From 00819cd4dd069d06065acba9c85bec26112603ff Mon Sep 17 00:00:00 2001 From: David Karlsson <35727626+dvdksn@users.noreply.github.com> Date: Thu, 11 Jun 2026 17:13:26 +0200 Subject: [PATCH 1/8] guides: revamp taxonomy, flatten structure, collapse multi-page series Phase 1 of the guides section revamp: - Replace 24-tag taxonomy with 7 focused use-case/SDLC tags: ai, testing, cicd, security, admin, databases, deployment - Remove language taxonomy entirely (delete data/languages.yaml, strip languages: front matter from all guides) - Delete 6 redirect shim pages (dhi-from-doi, dhi-from-wolfi, dhi-go-example, dhi-nodejs-example, dhi-python-example, lab-docker-for-ai-redirect) - Flatten 3 directory-wrapped single pages (bake, compose-bake, zscaler) - Retag all ~100 guide pages with new taxonomy - Move all tags: to params.tags for consistent placement - Collapse 47 multi-page guide series into single _index.md files (180 sub-pages merged; headings demoted one level, sub-page titles become H2 sections) - Add URL aliases for all deleted sub-pages so old URLs redirect to the parent guide Phase 2 (template rewrite: tag-pill UX, remove language filter, simplify guide cards) follows in a separate session. Co-Authored-By: Claude Sonnet 4.6 --- content/guides/_index.md | 6 + content/guides/admin-set-up/_index.md | 257 +- .../admin-set-up/comms-and-info-gathering.md | 75 - content/guides/admin-set-up/deploy.md | 36 - .../admin-set-up/finalize-plans-and-setup.md | 77 - content/guides/admin-set-up/testing.md | 61 - .../guides/admin-user-management/_index.md | 179 +- .../audit-and-monitor.md | 54 - .../guides/admin-user-management/onboard.md | 70 - content/guides/admin-user-management/setup.md | 47 - content/guides/agentic-ai.md | 2 +- content/guides/angular/_index.md | 1351 +++++++- .../angular/configure-github-actions.md | 323 -- content/guides/angular/containerize.md | 529 ---- content/guides/angular/deploy.md | 201 -- content/guides/angular/develop.md | 178 -- content/guides/angular/run-tests.md | 137 - content/guides/azure-pipelines.md | 4 +- content/guides/{bake/index.md => bake.md} | 23 +- content/guides/bun/_index.md | 504 ++- content/guides/bun/configure-ci-cd.md | 132 - content/guides/bun/containerize.md | 175 - content/guides/bun/deploy.md | 141 - content/guides/bun/develop.md | 75 - content/guides/claude-code-model-runner.md | 2 +- .../claude-code-sandbox-model-runner.md | 2 +- .../index.md => compose-bake.md} | 7 +- .../guides/container-supported-development.md | 5 +- content/guides/cpp/_index.md | 609 +++- content/guides/cpp/configure-ci-cd.md | 133 - content/guides/cpp/containerize.md | 93 - content/guides/cpp/deploy.md | 143 - content/guides/cpp/develop.md | 77 - content/guides/cpp/multistage.md | 112 - content/guides/cpp/security.md | 84 - content/guides/databases.md | 2 +- content/guides/deno/_index.md | 525 ++- content/guides/deno/configure-ci-cd.md | 131 - content/guides/deno/containerize.md | 197 -- content/guides/deno/deploy.md | 141 - content/guides/deno/develop.md | 75 - content/guides/dex.md | 3 +- content/guides/dhi-backstage.md | 2 +- content/guides/dhi-from-doi.md | 13 - content/guides/dhi-from-wolfi.md | 13 - content/guides/dhi-go-example.md | 14 - content/guides/dhi-nodejs-example.md | 15 - content/guides/dhi-openshift.md | 5 +- content/guides/dhi-python-example.md | 14 - content/guides/dhi-vex-walkthrough.md | 2 +- content/guides/django.md | 3 +- content/guides/docker-build-cloud/_index.md | 133 +- content/guides/docker-build-cloud/ci.md | 23 - .../docker-build-cloud/common-questions.md | 63 - content/guides/docker-build-cloud/dev.md | 19 - content/guides/docker-build-cloud/why.md | 28 - content/guides/docker-compose/_index.md | 125 +- .../guides/docker-compose/common-questions.md | 78 - content/guides/docker-compose/setup.md | 17 - content/guides/docker-compose/why.md | 23 - content/guides/docker-scout/_index.md | 276 +- content/guides/docker-scout/attestations.md | 36 - .../guides/docker-scout/common-questions.md | 61 - content/guides/docker-scout/demo.md | 22 - content/guides/docker-scout/remediation.md | 27 - content/guides/docker-scout/s3c.md | 68 - content/guides/docker-scout/sbom.md | 49 - content/guides/docker-scout/why.md | 28 - content/guides/dotnet/_index.md | 1280 +++++++- content/guides/dotnet/configure-ci-cd.md | 148 - content/guides/dotnet/containerize.md | 364 --- content/guides/dotnet/deploy.md | 224 -- content/guides/dotnet/develop.md | 425 --- content/guides/dotnet/run-tests.md | 149 - content/guides/frameworks/laravel/_index.md | 836 ++++- .../frameworks/laravel/common-questions.md | 38 - .../frameworks/laravel/development-setup.md | 328 -- .../frameworks/laravel/prerequisites.md | 26 - .../frameworks/laravel/production-setup.md | 444 --- .../claude-code-mcp-guide.md | 2 +- content/guides/genai-leveraging-rag/index.md | 2 +- content/guides/genai-pdf-bot/_index.md | 516 ++- content/guides/genai-pdf-bot/containerize.md | 264 -- content/guides/genai-pdf-bot/develop.md | 261 -- content/guides/genai-video-bot/index.md | 14 +- content/guides/gha.md | 5 +- .../guides/github-sonarqube-sandbox/_index.md | 2099 +++++++++++- .../github-sonarqube-sandbox/customize.md | 182 -- .../github-sonarqube-sandbox/troubleshoot.md | 334 -- .../github-sonarqube-sandbox/workflow.md | 1588 ---------- .../guides/go-prometheus-monitoring/_index.md | 591 +++- .../go-prometheus-monitoring/application.md | 250 -- .../go-prometheus-monitoring/compose.md | 166 - .../go-prometheus-monitoring/containerize.md | 103 - .../go-prometheus-monitoring/develop.md | 84 - content/guides/golang/_index.md | 1879 ++++++++++- content/guides/golang/build-images.md | 489 --- content/guides/golang/configure-ci-cd.md | 133 - content/guides/golang/deploy.md | 249 -- content/guides/golang/develop.md | 733 ----- content/guides/golang/run-containers.md | 210 -- content/guides/golang/run-tests.md | 96 - content/guides/grafana-mcp-server-gemini.md | 1 + content/guides/java/_index.md | 1089 ++++++- content/guides/java/configure-ci-cd.md | 143 - content/guides/java/containerize.md | 288 -- content/guides/java/deploy.md | 155 - content/guides/java/develop.md | 403 --- content/guides/java/run-tests.md | 125 - content/guides/jupyter.md | 3 +- content/guides/kafka.md | 3 +- content/guides/kube-deploy.md | 2 +- content/guides/lab-agentic-apps.md | 8 +- content/guides/lab-ai-fundamentals.md | 6 +- content/guides/lab-attestation-basics.md | 10 +- content/guides/lab-building-images.md | 10 +- content/guides/lab-compose-quickstart.md | 8 +- .../guides/lab-container-getting-started.md | 8 +- .../lab-container-supported-development.md | 8 +- content/guides/lab-containerized-sdlc.md | 8 +- .../lab-creating-ai-product-reviewer.md | 6 +- content/guides/lab-dhi-node.md | 10 +- content/guides/lab-docker-agent.md | 8 +- content/guides/lab-docker-for-ai-redirect.md | 9 - content/guides/lab-mcp-gateway.md | 8 +- content/guides/language-translation.md | 3 +- content/guides/localstack.md | 3 +- content/guides/named-entity-recognition.md | 3 +- content/guides/nextjs/_index.md | 1950 +++++++++++- .../guides/nextjs/configure-github-actions.md | 332 -- content/guides/nextjs/containerize.md | 961 ------ content/guides/nextjs/deploy.md | 199 -- content/guides/nextjs/develop.md | 209 -- content/guides/nextjs/run-tests.md | 267 -- content/guides/opencode-model-runner.md | 2 +- content/guides/opentelemetry.md | 3 +- content/guides/orchestration.md | 2 +- content/guides/pgadmin.md | 2 +- content/guides/php/_index.md | 1128 ++++++- content/guides/php/configure-ci-cd.md | 148 - content/guides/php/containerize.md | 283 -- content/guides/php/deploy.md | 146 - content/guides/php/develop.md | 458 --- content/guides/php/run-tests.md | 118 - content/guides/postgresql/_index.md | 1162 ++++++- ...vanced-configuration-and-initialization.md | 229 -- .../postgresql/companions-for-postgresql.md | 231 -- .../immediate-setup-and-data-persistence.md | 383 --- .../postgresql/networking-and-connectivity.md | 339 -- content/guides/pre-seeding.md | 14 +- content/guides/python/_index.md | 2812 ++++++++++++++++- .../guides/python/configure-github-actions.md | 161 - content/guides/python/containerize.md | 439 --- content/guides/python/deploy.md | 341 -- content/guides/python/develop.md | 1560 --------- content/guides/python/lint-format-typing.md | 196 -- content/guides/python/secure-supply-chain.md | 144 - content/guides/r/_index.md | 582 +++- content/guides/r/configure-ci-cd.md | 133 - content/guides/r/containerize.md | 100 - content/guides/r/deploy.md | 147 - content/guides/r/develop.md | 221 -- content/guides/rag-ollama/_index.md | 263 +- content/guides/rag-ollama/containerize.md | 108 - content/guides/rag-ollama/develop.md | 165 - content/guides/reactjs/_index.md | 1388 +++++++- .../reactjs/configure-github-actions.md | 321 -- content/guides/reactjs/containerize.md | 508 --- content/guides/reactjs/deploy.md | 194 -- content/guides/reactjs/develop.md | 205 -- content/guides/reactjs/run-tests.md | 179 -- content/guides/ros2/_index.md | 422 ++- content/guides/ros2/develop.md | 107 - content/guides/ros2/run-ros2.md | 77 - content/guides/ros2/turtlesim-example.md | 243 -- content/guides/ruby/_index.md | 853 ++++- .../guides/ruby/configure-github-actions.md | 111 - content/guides/ruby/containerize.md | 392 --- content/guides/ruby/deploy.md | 167 - content/guides/ruby/develop.md | 203 -- content/guides/rust/_index.md | 1157 ++++++- content/guides/rust/build-images.md | 310 -- content/guides/rust/configure-ci-cd.md | 132 - content/guides/rust/deploy.md | 238 -- content/guides/rust/develop.md | 303 -- content/guides/rust/run-containers.md | 198 -- content/guides/sentiment-analysis.md | 3 +- content/guides/swarm-deploy.md | 2 +- content/guides/tensorflowjs.md | 3 +- content/guides/testcontainers-cloud/_index.md | 111 +- .../testcontainers-cloud/common-questions.md | 44 - .../guides/testcontainers-cloud/demo-ci.md | 24 - .../guides/testcontainers-cloud/demo-local.md | 20 - content/guides/testcontainers-cloud/why.md | 19 - .../_index.md | 473 ++- .../create-project.md | 243 -- .../run-tests.md | 46 - .../write-tests.md | 189 -- .../_index.md | 237 +- .../create-project.md | 133 - .../run-tests.md | 41 - .../write-tests.md | 68 - .../_index.md | 385 ++- .../create-project.md | 104 - .../run-tests.md | 37 - .../test-suites.md | 145 - .../write-tests.md | 108 - .../_index.md | 424 ++- .../create-project.md | 242 -- .../run-tests.md | 38 - .../write-tests.md | 149 - .../_index.md | 299 +- .../create-project.md | 166 - .../run-tests.md | 43 - .../write-tests.md | 95 - .../testcontainers-java-jooq-flyway/_index.md | 582 +++- .../create-project.md | 332 -- .../run-tests.md | 38 - .../write-tests.md | 217 -- .../_index.md | 578 +++- .../create-project.md | 314 -- .../run-tests.md | 40 - .../write-tests.md | 229 -- .../testcontainers-java-lifecycle/_index.md | 441 ++- .../create-project.md | 167 - .../extension-annotations.md | 76 - .../lifecycle-callbacks.md | 93 - .../singleton-containers.md | 114 - .../_index.md | 446 ++- .../create-project.md | 283 -- .../run-tests.md | 41 - .../write-tests.md | 127 - .../_index.md | 646 +++- .../create-project.md | 215 -- .../run-tests.md | 44 - .../write-tests.md | 392 --- .../testcontainers-java-mockserver/_index.md | 419 ++- .../create-project.md | 217 -- .../run-tests.md | 40 - .../write-tests.md | 167 - .../testcontainers-java-quarkus/_index.md | 445 ++- .../create-project.md | 192 -- .../testcontainers-java-quarkus/run-tests.md | 72 - .../write-tests.md | 186 -- .../testcontainers-java-replace-h2/_index.md | 239 +- .../jdbc-url-approach.md | 101 - .../junit-extension-approach.md | 81 - .../problem-with-h2.md | 62 - .../_index.md | 206 +- .../copy-files.md | 89 - .../exec-in-container.md | 118 - .../_index.md | 445 ++- .../create-project.md | 296 -- .../run-tests.md | 40 - .../write-tests.md | 114 - .../_index.md | 316 +- .../create-project.md | 182 -- .../run-tests.md | 38 - .../write-tests.md | 101 - .../testcontainers-java-wiremock/_index.md | 743 ++++- .../create-project.md | 208 -- .../testcontainers-java-wiremock/run-tests.md | 44 - .../write-tests.md | 496 --- .../_index.md | 173 +- .../create-project.md | 54 - .../run-tests.md | 61 - .../write-tests.md | 63 - .../_index.md | 283 +- .../create-project.md | 131 - .../run-tests.md | 52 - .../write-tests.md | 105 - content/guides/text-classification.md | 3 +- content/guides/text-summarization.md | 3 +- content/guides/traefik.md | 2 +- content/guides/vuejs/_index.md | 1360 +++++++- .../guides/vuejs/configure-github-actions.md | 320 -- content/guides/vuejs/containerize.md | 530 ---- content/guides/vuejs/deploy.md | 201 -- content/guides/vuejs/develop.md | 189 -- content/guides/vuejs/run-tests.md | 138 - content/guides/wiremock.md | 3 +- .../guides/{zscaler/index.md => zscaler.md} | 4 +- data/languages.yaml | 30 - data/tags.yaml | 44 +- 284 files changed, 33084 insertions(+), 34249 deletions(-) delete mode 100644 content/guides/admin-set-up/comms-and-info-gathering.md delete mode 100644 content/guides/admin-set-up/deploy.md delete mode 100644 content/guides/admin-set-up/finalize-plans-and-setup.md delete mode 100644 content/guides/admin-set-up/testing.md delete mode 100644 content/guides/admin-user-management/audit-and-monitor.md delete mode 100644 content/guides/admin-user-management/onboard.md delete mode 100644 content/guides/admin-user-management/setup.md delete mode 100644 content/guides/angular/configure-github-actions.md delete mode 100644 content/guides/angular/containerize.md delete mode 100644 content/guides/angular/deploy.md delete mode 100644 content/guides/angular/develop.md delete mode 100644 content/guides/angular/run-tests.md rename content/guides/{bake/index.md => bake.md} (97%) delete mode 100644 content/guides/bun/configure-ci-cd.md delete mode 100644 content/guides/bun/containerize.md delete mode 100644 content/guides/bun/deploy.md delete mode 100644 content/guides/bun/develop.md rename content/guides/{compose-bake/index.md => compose-bake.md} (98%) delete mode 100644 content/guides/cpp/configure-ci-cd.md delete mode 100644 content/guides/cpp/containerize.md delete mode 100644 content/guides/cpp/deploy.md delete mode 100644 content/guides/cpp/develop.md delete mode 100644 content/guides/cpp/multistage.md delete mode 100644 content/guides/cpp/security.md delete mode 100644 content/guides/deno/configure-ci-cd.md delete mode 100644 content/guides/deno/containerize.md delete mode 100644 content/guides/deno/deploy.md delete mode 100644 content/guides/deno/develop.md delete mode 100644 content/guides/dhi-from-doi.md delete mode 100644 content/guides/dhi-from-wolfi.md delete mode 100644 content/guides/dhi-go-example.md delete mode 100644 content/guides/dhi-nodejs-example.md delete mode 100644 content/guides/dhi-python-example.md delete mode 100644 content/guides/docker-build-cloud/ci.md delete mode 100644 content/guides/docker-build-cloud/common-questions.md delete mode 100644 content/guides/docker-build-cloud/dev.md delete mode 100644 content/guides/docker-build-cloud/why.md delete mode 100644 content/guides/docker-compose/common-questions.md delete mode 100644 content/guides/docker-compose/setup.md delete mode 100644 content/guides/docker-compose/why.md delete mode 100644 content/guides/docker-scout/attestations.md delete mode 100644 content/guides/docker-scout/common-questions.md delete mode 100644 content/guides/docker-scout/demo.md delete mode 100644 content/guides/docker-scout/remediation.md delete mode 100644 content/guides/docker-scout/s3c.md delete mode 100644 content/guides/docker-scout/sbom.md delete mode 100644 content/guides/docker-scout/why.md delete mode 100644 content/guides/dotnet/configure-ci-cd.md delete mode 100644 content/guides/dotnet/containerize.md delete mode 100644 content/guides/dotnet/deploy.md delete mode 100644 content/guides/dotnet/develop.md delete mode 100644 content/guides/dotnet/run-tests.md delete mode 100644 content/guides/frameworks/laravel/common-questions.md delete mode 100644 content/guides/frameworks/laravel/development-setup.md delete mode 100644 content/guides/frameworks/laravel/prerequisites.md delete mode 100644 content/guides/frameworks/laravel/production-setup.md delete mode 100644 content/guides/genai-pdf-bot/containerize.md delete mode 100644 content/guides/genai-pdf-bot/develop.md delete mode 100644 content/guides/github-sonarqube-sandbox/customize.md delete mode 100644 content/guides/github-sonarqube-sandbox/troubleshoot.md delete mode 100644 content/guides/github-sonarqube-sandbox/workflow.md delete mode 100644 content/guides/go-prometheus-monitoring/application.md delete mode 100644 content/guides/go-prometheus-monitoring/compose.md delete mode 100644 content/guides/go-prometheus-monitoring/containerize.md delete mode 100644 content/guides/go-prometheus-monitoring/develop.md delete mode 100644 content/guides/golang/build-images.md delete mode 100644 content/guides/golang/configure-ci-cd.md delete mode 100644 content/guides/golang/deploy.md delete mode 100644 content/guides/golang/develop.md delete mode 100644 content/guides/golang/run-containers.md delete mode 100644 content/guides/golang/run-tests.md delete mode 100644 content/guides/java/configure-ci-cd.md delete mode 100644 content/guides/java/containerize.md delete mode 100644 content/guides/java/deploy.md delete mode 100644 content/guides/java/develop.md delete mode 100644 content/guides/java/run-tests.md delete mode 100644 content/guides/lab-docker-for-ai-redirect.md delete mode 100644 content/guides/nextjs/configure-github-actions.md delete mode 100644 content/guides/nextjs/containerize.md delete mode 100644 content/guides/nextjs/deploy.md delete mode 100644 content/guides/nextjs/develop.md delete mode 100644 content/guides/nextjs/run-tests.md delete mode 100644 content/guides/php/configure-ci-cd.md delete mode 100644 content/guides/php/containerize.md delete mode 100644 content/guides/php/deploy.md delete mode 100644 content/guides/php/develop.md delete mode 100644 content/guides/php/run-tests.md delete mode 100644 content/guides/postgresql/advanced-configuration-and-initialization.md delete mode 100644 content/guides/postgresql/companions-for-postgresql.md delete mode 100644 content/guides/postgresql/immediate-setup-and-data-persistence.md delete mode 100644 content/guides/postgresql/networking-and-connectivity.md delete mode 100644 content/guides/python/configure-github-actions.md delete mode 100644 content/guides/python/containerize.md delete mode 100644 content/guides/python/deploy.md delete mode 100644 content/guides/python/develop.md delete mode 100644 content/guides/python/lint-format-typing.md delete mode 100644 content/guides/python/secure-supply-chain.md delete mode 100644 content/guides/r/configure-ci-cd.md delete mode 100644 content/guides/r/containerize.md delete mode 100644 content/guides/r/deploy.md delete mode 100644 content/guides/r/develop.md delete mode 100644 content/guides/rag-ollama/containerize.md delete mode 100644 content/guides/rag-ollama/develop.md delete mode 100644 content/guides/reactjs/configure-github-actions.md delete mode 100644 content/guides/reactjs/containerize.md delete mode 100644 content/guides/reactjs/deploy.md delete mode 100644 content/guides/reactjs/develop.md delete mode 100644 content/guides/reactjs/run-tests.md delete mode 100644 content/guides/ros2/develop.md delete mode 100644 content/guides/ros2/run-ros2.md delete mode 100644 content/guides/ros2/turtlesim-example.md delete mode 100644 content/guides/ruby/configure-github-actions.md delete mode 100644 content/guides/ruby/containerize.md delete mode 100644 content/guides/ruby/deploy.md delete mode 100644 content/guides/ruby/develop.md delete mode 100644 content/guides/rust/build-images.md delete mode 100644 content/guides/rust/configure-ci-cd.md delete mode 100644 content/guides/rust/deploy.md delete mode 100644 content/guides/rust/develop.md delete mode 100644 content/guides/rust/run-containers.md delete mode 100644 content/guides/testcontainers-cloud/common-questions.md delete mode 100644 content/guides/testcontainers-cloud/demo-ci.md delete mode 100644 content/guides/testcontainers-cloud/demo-local.md delete mode 100644 content/guides/testcontainers-cloud/why.md delete mode 100644 content/guides/testcontainers-dotnet-aspnet-core/create-project.md delete mode 100644 content/guides/testcontainers-dotnet-aspnet-core/run-tests.md delete mode 100644 content/guides/testcontainers-dotnet-aspnet-core/write-tests.md delete mode 100644 content/guides/testcontainers-dotnet-getting-started/create-project.md delete mode 100644 content/guides/testcontainers-dotnet-getting-started/run-tests.md delete mode 100644 content/guides/testcontainers-dotnet-getting-started/write-tests.md delete mode 100644 content/guides/testcontainers-go-getting-started/create-project.md delete mode 100644 content/guides/testcontainers-go-getting-started/run-tests.md delete mode 100644 content/guides/testcontainers-go-getting-started/test-suites.md delete mode 100644 content/guides/testcontainers-go-getting-started/write-tests.md delete mode 100644 content/guides/testcontainers-java-aws-localstack/create-project.md delete mode 100644 content/guides/testcontainers-java-aws-localstack/run-tests.md delete mode 100644 content/guides/testcontainers-java-aws-localstack/write-tests.md delete mode 100644 content/guides/testcontainers-java-getting-started/create-project.md delete mode 100644 content/guides/testcontainers-java-getting-started/run-tests.md delete mode 100644 content/guides/testcontainers-java-getting-started/write-tests.md delete mode 100644 content/guides/testcontainers-java-jooq-flyway/create-project.md delete mode 100644 content/guides/testcontainers-java-jooq-flyway/run-tests.md delete mode 100644 content/guides/testcontainers-java-jooq-flyway/write-tests.md delete mode 100644 content/guides/testcontainers-java-keycloak-spring-boot/create-project.md delete mode 100644 content/guides/testcontainers-java-keycloak-spring-boot/run-tests.md delete mode 100644 content/guides/testcontainers-java-keycloak-spring-boot/write-tests.md delete mode 100644 content/guides/testcontainers-java-lifecycle/create-project.md delete mode 100644 content/guides/testcontainers-java-lifecycle/extension-annotations.md delete mode 100644 content/guides/testcontainers-java-lifecycle/lifecycle-callbacks.md delete mode 100644 content/guides/testcontainers-java-lifecycle/singleton-containers.md delete mode 100644 content/guides/testcontainers-java-micronaut-kafka/create-project.md delete mode 100644 content/guides/testcontainers-java-micronaut-kafka/run-tests.md delete mode 100644 content/guides/testcontainers-java-micronaut-kafka/write-tests.md delete mode 100644 content/guides/testcontainers-java-micronaut-wiremock/create-project.md delete mode 100644 content/guides/testcontainers-java-micronaut-wiremock/run-tests.md delete mode 100644 content/guides/testcontainers-java-micronaut-wiremock/write-tests.md delete mode 100644 content/guides/testcontainers-java-mockserver/create-project.md delete mode 100644 content/guides/testcontainers-java-mockserver/run-tests.md delete mode 100644 content/guides/testcontainers-java-mockserver/write-tests.md delete mode 100644 content/guides/testcontainers-java-quarkus/create-project.md delete mode 100644 content/guides/testcontainers-java-quarkus/run-tests.md delete mode 100644 content/guides/testcontainers-java-quarkus/write-tests.md delete mode 100644 content/guides/testcontainers-java-replace-h2/jdbc-url-approach.md delete mode 100644 content/guides/testcontainers-java-replace-h2/junit-extension-approach.md delete mode 100644 content/guides/testcontainers-java-replace-h2/problem-with-h2.md delete mode 100644 content/guides/testcontainers-java-service-configuration/copy-files.md delete mode 100644 content/guides/testcontainers-java-service-configuration/exec-in-container.md delete mode 100644 content/guides/testcontainers-java-spring-boot-kafka/create-project.md delete mode 100644 content/guides/testcontainers-java-spring-boot-kafka/run-tests.md delete mode 100644 content/guides/testcontainers-java-spring-boot-kafka/write-tests.md delete mode 100644 content/guides/testcontainers-java-spring-boot-rest-api/create-project.md delete mode 100644 content/guides/testcontainers-java-spring-boot-rest-api/run-tests.md delete mode 100644 content/guides/testcontainers-java-spring-boot-rest-api/write-tests.md delete mode 100644 content/guides/testcontainers-java-wiremock/create-project.md delete mode 100644 content/guides/testcontainers-java-wiremock/run-tests.md delete mode 100644 content/guides/testcontainers-java-wiremock/write-tests.md delete mode 100644 content/guides/testcontainers-nodejs-getting-started/create-project.md delete mode 100644 content/guides/testcontainers-nodejs-getting-started/run-tests.md delete mode 100644 content/guides/testcontainers-nodejs-getting-started/write-tests.md delete mode 100644 content/guides/testcontainers-python-getting-started/create-project.md delete mode 100644 content/guides/testcontainers-python-getting-started/run-tests.md delete mode 100644 content/guides/testcontainers-python-getting-started/write-tests.md delete mode 100644 content/guides/vuejs/configure-github-actions.md delete mode 100644 content/guides/vuejs/containerize.md delete mode 100644 content/guides/vuejs/deploy.md delete mode 100644 content/guides/vuejs/develop.md delete mode 100644 content/guides/vuejs/run-tests.md rename content/guides/{zscaler/index.md => zscaler.md} (98%) delete mode 100644 data/languages.yaml diff --git a/content/guides/_index.md b/content/guides/_index.md index b5ded5dff3eb..100bdaf9b24a 100644 --- a/content/guides/_index.md +++ b/content/guides/_index.md @@ -9,6 +9,12 @@ layout: landing aliases: - /guides/language/ - /language/ + - /guides/dhi-from-doi/ + - /guides/dhi-from-wolfi/ + - /guides/dhi-go-example/ + - /guides/dhi-nodejs-example/ + - /guides/dhi-python-example/ + - /guides/lab-docker-for-ai-redirect/ - /learning-paths/ --- diff --git a/content/guides/admin-set-up/_index.md b/content/guides/admin-set-up/_index.md index e20eb016eecb..e77cb481df49 100644 --- a/content/guides/admin-set-up/_index.md +++ b/content/guides/admin-set-up/_index.md @@ -4,29 +4,19 @@ linkTitle: Admin set up summary: Get the most out of Docker by streamlining workflows, standardizing development environments, and ensuring smooth deployments across your company. description: Learn how to onboard your company and take advantage of all of the Docker products and features. keywords: admin, onboarding, deployment, organization setup, docker business, rollout -tags: [admin] +aliases: + - /guides/admin-set-up/comms-and-info-gathering/ + - /guides/admin-set-up/deploy/ + - /guides/admin-set-up/finalize-plans-and-setup/ + - /guides/admin-set-up/testing/ params: + tags: [admin] time: 20 minutes image: - resource_links: - - title: Overview of Administration in Docker - url: /admin/ - - title: Single sign-on - url: /security/for-admins/single-sign-on/ - - title: Enforce sign-in - url: /security/for-admins/enforce-sign-in/ - - title: Roles and permissions - url: /security/for-admins/roles-and-permissions/ - - title: Settings Management - url: /security/for-admins/hardened-desktop/settings-management/ - - title: Registry Access Management - url: /security/for-admins/hardened-desktop/registry-access-management/ - - title: Image Access Management - url: /security/for-admins/hardened-desktop/image-access-management/ - - title: Docker subscription information url: "https://www.docker.com/pricing?ref=Docs&refAction=DocsGuidesAdminSetup" --- + Docker's tools provide a scalable, secure platform that empowers your developers to create, ship, and run applications faster. As an administrator, you can streamline workflows, standardize development environments, and ensure @@ -106,3 +96,236 @@ This guide covers integration with: - Entra ID SAML 2.0 - Azure Connect (OIDC) - MDM solutions like Intune + +## Communication and information gathering + +### Communicate with your developers and IT teams + +Before rolling out Docker Desktop across your organization, coordinate with key stakeholders to ensure a smooth transition. + +#### Notify Docker Desktop users + +You may already have Docker Desktop users within your company. Some steps in +this onboarding process may affect how they interact with the platform. + +Communicate early with users to inform them that: + +- They'll be upgraded to a supported version of Docker Desktop as part of the subscription onboarding +- Settings will be reviewed and optimized for productivity +- They'll need to sign in to the company's Docker organization using their + business email to access subscription benefits + +#### Engage with your MDM team + +Device management solutions, such as Intune and Jamf, are commonly used for +software distribution across enterprises. These tools are typically managed by a dedicated MDM team. + +Engage with this team early in the process to: + +- Understand their requirements and lead time for deploying changes +- Coordinate the distribution of configuration files + +Several setup steps in this guide require JSON files, registry keys, or .plist +files to be distributed to developer machines. Use MDM tools to deploy these configuration files and ensure their integrity. + +### Identify Docker organizations + +Some companies may have more than one +[Docker organization](/manuals/admin/organization/_index.md) created. These +organizations may have been created for specific purposes, or may not be +needed anymore. + +If you suspect your company has multiple Docker organizations: + +- Survey your teams to see if they have their own organizations +- Contact your Docker Support to get a list of organizations with users whose + emails match your domain name + +### Gather requirements + +[Settings Management](/manuals/enterprise/security/hardened-desktop/settings-management/_index.md) lets you preset numerous configuration parameters for Docker Desktop. + +Work with the following stakeholders to establish your company's baseline +configuration: + +- Docker organization owner +- Development lead +- Information security representative + +Review these areas together: + +- Security features and + [enforcing sign-in](/manuals/enterprise/security/enforce-sign-in/_index.md) + for Docker Desktop users +- Additional Docker products included in your subscriptions + +To view the parameters that can be preset, see [Configure Settings Management](/manuals/enterprise/security/hardened-desktop/settings-management/configure-json-file.md#step-two-configure-the-settings-you-want-to-lock-in). + +### Optional: Meet with the Docker Implementation team + +The Docker Implementation team can help you set up your organization, +configure SSO, enforce sign-in, and configure Docker Desktop. + +To schedule a meeting, email successteam@docker.com. + +## Finalize plans and begin setup + +### Send finalized settings files to the MDM team + +After reaching an agreement with the relevant teams about your baseline and +security configurations as outlined in the previous section, configure Settings Management using either the [Docker Admin Console](/manuals/enterprise/security/hardened-desktop/settings-management/configure-admin-console.md) or an +[`admin-settings.json` file](/manuals/enterprise/security/hardened-desktop/settings-management/configure-json-file.md). + +Once the file is ready, collaborate with your MDM team to deploy your chosen +settings, along with your chosen method for [enforcing sign-in](/manuals/enterprise/security/enforce-sign-in/_index.md). + +> [!IMPORTANT] +> +> Test this first with a small number of Docker Desktop developers to verify the functionality works as expected before deploying more widely. + +### Manage your organizations + +If you have more than one organization, consider either [consolidating them +into one organization](/manuals/admin/organization/setup/orgs.md) or creating a +[Docker company](/manuals/admin/company/_index.md) to manage multiple +organizations. + +### Begin setup + +#### Set up single sign-on and domain verification + +Single sign-on (SSO) lets developers authenticate using their identity +providers (IdPs) to access Docker. SSO is available for a whole company and all associated organizations, or an individual organization that has a Docker +Business subscription. For more information, see the +[documentation](/manuals/enterprise/security/single-sign-on/_index.md). + +You can also enable [SCIM](/manuals/enterprise/security/provisioning/scim/_index.md) +for further automation of provisioning and deprovisioning of users. + +#### Set up Docker product entitlements included in the subscription + +[Docker Build Cloud](/manuals/build-cloud/_index.md) significantly reduces +build times, both locally and in CI, by providing a dedicated remote builder +and shared cache. Powered by the cloud, developer time and local resources are +freed up so your team can focus on more important things, like innovation. +To get started, [set up a cloud builder](https://app.docker.com/build/). + +[Docker Scout](manuals/scout/_index.md) is a solution for proactively enhancing +your software supply chain security. By analyzing your images, Docker Scout +compiles an inventory of components, also known as a Software Bill of Materials +(SBOM). The SBOM is matched against a continuously updated vulnerability +database to pinpoint security weaknesses. To get started, see +[Quickstart](/manuals/scout/quickstart.md). + +[Testcontainers Cloud](https://testcontainers.com/cloud/docs/) allows +developers to run containers in the cloud, removing the need to run heavy +containers on your local machine. + +[Docker Hardened Images](/manuals/dhi/_index.md) are minimal, secure, and production-ready container base and application images maintained by Docker. +Designed to reduce vulnerabilities and simplify compliance, DHIs integrate +easily into your existing Docker-based workflows with little to no retooling +required. + +#### Ensure you're running a supported version of Docker Desktop + +> [!WARNING] +> +> This step could affect the experience for users on older versions of Docker +> Desktop. + +Existing users may be running outdated or unsupported versions of +Docker Desktop. All users should update to a supported version. Docker Desktop +versions released within the past 6 months from the latest release are supported. + +Use an MDM solution to manage the version of Docker Desktop for users. Users +may also get Docker Desktop directly from Docker or through a company software +portal. + +## Testing + +### SSO and SCIM testing + +Test SSO and SCIM by signing in to Docker Desktop or Docker Hub with the email +address linked to a Docker account that is part of the verified domain. +Developers who sign in using their Docker usernames remain unaffected by the +SSO and SCIM setup. + +> [!IMPORTANT] +> +> Some users may need CLI based logins to Docker Hub, and for this they will +> need a [personal access token (PAT)](/manuals/security/access-tokens.md). + +### Test Registry Access Management and Image Access Management + +> [!WARNING] +> +> Communicate with your users before proceeding, as this step will impact all +> existing users signing into your Docker organization. + +If you plan to use [Registry Access Management (RAM)](/manuals/enterprise/security/hardened-desktop/registry-access-management.md) and/or [Image Access Management (IAM)](/manuals/enterprise/security/hardened-desktop/image-access-management.md): + +1. Ensure your test developer signs in to Docker Desktop using their + organization credentials +2. Have them attempt to pull an unauthorized image or one from a disallowed + registry via the Docker CLI +3. Verify they receive an error message indicating that the registry is + restricted by the organization + +### Deploy settings and enforce sign in to test group + +Deploy the Docker settings and enforce sign-in for a small group of test users +via MDM. Have this group test their development workflows with containers on +Docker Desktop and Docker Hub to ensure all settings and the sign-in enforcement +function as expected. + +### Test Docker Build Cloud capabilities + +Have one of your Docker Desktop testers [connect to the cloud builder you created and use it to build](/manuals/build-cloud/usage.md). + +### Test Testcontainers Cloud + +Have a test developer [connect to Testcontainers Cloud](https://testcontainers.com/cloud/docs/#getting-started) and run a container in +the cloud to verify the setup is working correctly. + +### Verify Docker Scout monitoring of repositories + +Check the [Docker Scout dashboard](https://scout.docker.com/) to confirm that +data is being properly received for the repositories where Docker Scout has +been enabled. + +### Verify access to Docker Hardened Images + +Have a test developer attempt to [pull a Docker Hardened Image](/manuals/dhi/get-started.md) to confirm that +the team has proper access and can integrate these images into their workflows. + +## Deploy your Docker setup + +> [!WARNING] +> +> Communicate with your users before proceeding, and confirm that your IT and +> MDM teams are prepared to handle any unexpected issues, as these steps will +> affect all existing users signing into your Docker organization. + +### Enforce SSO + +Enforcing SSO means that anyone who has a Docker profile with an email address +that matches your verified domain must sign in using your SSO connection. Make +sure the Identity provider groups associated with your SSO connection cover all +the developer groups that you want to have access to the Docker subscription. + +For instructions on how to enforce SSO, see [Enforce SSO](/manuals/enterprise/security/single-sign-on/connect.md). + +### Deploy configuration settings and enforce sign-in to users + +Have the MDM team deploy the configuration files for Docker to all users. + +### Next steps + +Congratulations, you've successfully completed the admin implementation process +for Docker. + +To continue optimizing your Docker environment: + +- Review your [organization's usage data](/manuals/admin/insights.md) to track adoption +- Monitor [Docker Scout findings](/manuals/scout/explore/analysis.md) for security insights +- Explore [additional security features](/manuals/enterprise/security/_index.md) to enhance your configuration diff --git a/content/guides/admin-set-up/comms-and-info-gathering.md b/content/guides/admin-set-up/comms-and-info-gathering.md deleted file mode 100644 index 40a230c5e36c..000000000000 --- a/content/guides/admin-set-up/comms-and-info-gathering.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -title: Communication and information gathering -description: Gather your company's requirements from key stakeholders and communicate to your developers. -keywords: admin, onboarding, stakeholders, communication, mdm, requirements -weight: 10 ---- - -## Communicate with your developers and IT teams - -Before rolling out Docker Desktop across your organization, coordinate with key stakeholders to ensure a smooth transition. - -### Notify Docker Desktop users - -You may already have Docker Desktop users within your company. Some steps in -this onboarding process may affect how they interact with the platform. - -Communicate early with users to inform them that: - -- They'll be upgraded to a supported version of Docker Desktop as part of the subscription onboarding -- Settings will be reviewed and optimized for productivity -- They'll need to sign in to the company's Docker organization using their - business email to access subscription benefits - -### Engage with your MDM team - -Device management solutions, such as Intune and Jamf, are commonly used for -software distribution across enterprises. These tools are typically managed by a dedicated MDM team. - -Engage with this team early in the process to: - -- Understand their requirements and lead time for deploying changes -- Coordinate the distribution of configuration files - -Several setup steps in this guide require JSON files, registry keys, or .plist -files to be distributed to developer machines. Use MDM tools to deploy these configuration files and ensure their integrity. - -## Identify Docker organizations - -Some companies may have more than one -[Docker organization](/manuals/admin/organization/_index.md) created. These -organizations may have been created for specific purposes, or may not be -needed anymore. - -If you suspect your company has multiple Docker organizations: - -- Survey your teams to see if they have their own organizations -- Contact your Docker Support to get a list of organizations with users whose - emails match your domain name - -## Gather requirements - -[Settings Management](/manuals/enterprise/security/hardened-desktop/settings-management/_index.md) lets you preset numerous configuration parameters for Docker Desktop. - -Work with the following stakeholders to establish your company's baseline -configuration: - -- Docker organization owner -- Development lead -- Information security representative - -Review these areas together: - -- Security features and - [enforcing sign-in](/manuals/enterprise/security/enforce-sign-in/_index.md) - for Docker Desktop users -- Additional Docker products included in your subscriptions - -To view the parameters that can be preset, see [Configure Settings Management](/manuals/enterprise/security/hardened-desktop/settings-management/configure-json-file.md#step-two-configure-the-settings-you-want-to-lock-in). - -## Optional: Meet with the Docker Implementation team - -The Docker Implementation team can help you set up your organization, -configure SSO, enforce sign-in, and configure Docker Desktop. - -To schedule a meeting, email successteam@docker.com. diff --git a/content/guides/admin-set-up/deploy.md b/content/guides/admin-set-up/deploy.md deleted file mode 100644 index 30f202417b08..000000000000 --- a/content/guides/admin-set-up/deploy.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -title: Deploy your Docker setup -description: Deploy your Docker setup across your company. -keywords: admin, onboarding, deployment, sso, rollout, organization -weight: 40 ---- - -> [!WARNING] -> -> Communicate with your users before proceeding, and confirm that your IT and -> MDM teams are prepared to handle any unexpected issues, as these steps will -> affect all existing users signing into your Docker organization. - -## Enforce SSO - -Enforcing SSO means that anyone who has a Docker profile with an email address -that matches your verified domain must sign in using your SSO connection. Make -sure the Identity provider groups associated with your SSO connection cover all -the developer groups that you want to have access to the Docker subscription. - -For instructions on how to enforce SSO, see [Enforce SSO](/manuals/enterprise/security/single-sign-on/connect.md). - -## Deploy configuration settings and enforce sign-in to users - -Have the MDM team deploy the configuration files for Docker to all users. - -## Next steps - -Congratulations, you've successfully completed the admin implementation process -for Docker. - -To continue optimizing your Docker environment: - -- Review your [organization's usage data](/manuals/admin/insights.md) to track adoption -- Monitor [Docker Scout findings](/manuals/scout/explore/analysis.md) for security insights -- Explore [additional security features](/manuals/enterprise/security/_index.md) to enhance your configuration diff --git a/content/guides/admin-set-up/finalize-plans-and-setup.md b/content/guides/admin-set-up/finalize-plans-and-setup.md deleted file mode 100644 index d31ec79b3082..000000000000 --- a/content/guides/admin-set-up/finalize-plans-and-setup.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -title: Finalize plans and begin setup -description: Collaborate with your MDM team to distribute configurations and set up SSO and Docker product trials. -keywords: admin, onboarding, mdm, settings management, sso, configuration -weight: 20 ---- - -## Send finalized settings files to the MDM team - -After reaching an agreement with the relevant teams about your baseline and -security configurations as outlined in the previous section, configure Settings Management using either the [Docker Admin Console](/manuals/enterprise/security/hardened-desktop/settings-management/configure-admin-console.md) or an -[`admin-settings.json` file](/manuals/enterprise/security/hardened-desktop/settings-management/configure-json-file.md). - -Once the file is ready, collaborate with your MDM team to deploy your chosen -settings, along with your chosen method for [enforcing sign-in](/manuals/enterprise/security/enforce-sign-in/_index.md). - -> [!IMPORTANT] -> -> Test this first with a small number of Docker Desktop developers to verify the functionality works as expected before deploying more widely. - -## Manage your organizations - -If you have more than one organization, consider either [consolidating them -into one organization](/manuals/admin/organization/setup/orgs.md) or creating a -[Docker company](/manuals/admin/company/_index.md) to manage multiple -organizations. - -## Begin setup - -### Set up single sign-on and domain verification - -Single sign-on (SSO) lets developers authenticate using their identity -providers (IdPs) to access Docker. SSO is available for a whole company and all associated organizations, or an individual organization that has a Docker -Business subscription. For more information, see the -[documentation](/manuals/enterprise/security/single-sign-on/_index.md). - -You can also enable [SCIM](/manuals/enterprise/security/provisioning/scim/_index.md) -for further automation of provisioning and deprovisioning of users. - -### Set up Docker product entitlements included in the subscription - -[Docker Build Cloud](/manuals/build-cloud/_index.md) significantly reduces -build times, both locally and in CI, by providing a dedicated remote builder -and shared cache. Powered by the cloud, developer time and local resources are -freed up so your team can focus on more important things, like innovation. -To get started, [set up a cloud builder](https://app.docker.com/build/). - -[Docker Scout](manuals/scout/_index.md) is a solution for proactively enhancing -your software supply chain security. By analyzing your images, Docker Scout -compiles an inventory of components, also known as a Software Bill of Materials -(SBOM). The SBOM is matched against a continuously updated vulnerability -database to pinpoint security weaknesses. To get started, see -[Quickstart](/manuals/scout/quickstart.md). - -[Testcontainers Cloud](https://testcontainers.com/cloud/docs/) allows -developers to run containers in the cloud, removing the need to run heavy -containers on your local machine. - -[Docker Hardened Images](/manuals/dhi/_index.md) are minimal, secure, and production-ready container base and application images maintained by Docker. -Designed to reduce vulnerabilities and simplify compliance, DHIs integrate -easily into your existing Docker-based workflows with little to no retooling -required. - -### Ensure you're running a supported version of Docker Desktop - -> [!WARNING] -> -> This step could affect the experience for users on older versions of Docker -> Desktop. - -Existing users may be running outdated or unsupported versions of -Docker Desktop. All users should update to a supported version. Docker Desktop -versions released within the past 6 months from the latest release are supported. - -Use an MDM solution to manage the version of Docker Desktop for users. Users -may also get Docker Desktop directly from Docker or through a company software -portal. diff --git a/content/guides/admin-set-up/testing.md b/content/guides/admin-set-up/testing.md deleted file mode 100644 index a51b4c087e92..000000000000 --- a/content/guides/admin-set-up/testing.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -title: Testing -description: Test your Docker setup. -keywords: admin, onboarding, testing, sso, scim, verification -weight: 30 ---- - -## SSO and SCIM testing - -Test SSO and SCIM by signing in to Docker Desktop or Docker Hub with the email -address linked to a Docker account that is part of the verified domain. -Developers who sign in using their Docker usernames remain unaffected by the -SSO and SCIM setup. - -> [!IMPORTANT] -> -> Some users may need CLI based logins to Docker Hub, and for this they will -> need a [personal access token (PAT)](/manuals/security/access-tokens.md). - -## Test Registry Access Management and Image Access Management - -> [!WARNING] -> -> Communicate with your users before proceeding, as this step will impact all -> existing users signing into your Docker organization. - -If you plan to use [Registry Access Management (RAM)](/manuals/enterprise/security/hardened-desktop/registry-access-management.md) and/or [Image Access Management (IAM)](/manuals/enterprise/security/hardened-desktop/image-access-management.md): - -1. Ensure your test developer signs in to Docker Desktop using their - organization credentials -2. Have them attempt to pull an unauthorized image or one from a disallowed - registry via the Docker CLI -3. Verify they receive an error message indicating that the registry is - restricted by the organization - -## Deploy settings and enforce sign in to test group - -Deploy the Docker settings and enforce sign-in for a small group of test users -via MDM. Have this group test their development workflows with containers on -Docker Desktop and Docker Hub to ensure all settings and the sign-in enforcement -function as expected. - -## Test Docker Build Cloud capabilities - -Have one of your Docker Desktop testers [connect to the cloud builder you created and use it to build](/manuals/build-cloud/usage.md). - -## Test Testcontainers Cloud - -Have a test developer [connect to Testcontainers Cloud](https://testcontainers.com/cloud/docs/#getting-started) and run a container in -the cloud to verify the setup is working correctly. - -## Verify Docker Scout monitoring of repositories - -Check the [Docker Scout dashboard](https://scout.docker.com/) to confirm that -data is being properly received for the repositories where Docker Scout has -been enabled. - -## Verify access to Docker Hardened Images - -Have a test developer attempt to [pull a Docker Hardened Image](/manuals/dhi/get-started.md) to confirm that -the team has proper access and can integrate these images into their workflows. diff --git a/content/guides/admin-user-management/_index.md b/content/guides/admin-user-management/_index.md index 7fe711c9daf5..84ff0adb7545 100644 --- a/content/guides/admin-user-management/_index.md +++ b/content/guides/admin-user-management/_index.md @@ -3,26 +3,18 @@ title: Mastering user and access management summary: Simplify user access while ensuring security and efficiency in Docker. description: A guide for managing roles, provisioning users, and optimizing Docker access with tools like SSO and activity logs. keywords: admin, user management, roles, permissions, sso, provisioning, access control -tags: [admin] +aliases: + - /guides/admin-user-management/audit-and-monitor/ + - /guides/admin-user-management/onboard/ + - /guides/admin-user-management/setup/ params: + tags: [admin] featured: false time: 20 minutes image: - resource_links: - - title: Overview of Administration in Docker - url: /admin/ - - title: Single sign-on - url: /security/for-admins/single-sign-on/ - - title: Onboard your organization - url: /admin/organization/setup/onboard/ - - title: Roles and permissions - url: /security/for-admins/roles-and-permissions/ - - title: Insights - url: /admin/insights/ - - title: Activity logs - url: /admin/activity-logs/ --- + Managing roles and permissions is key to securing your Docker environment while enabling easy collaboration and operational efficiency. This guide walks IT administrators through the essentials of user and access management, offering strategies for assigning roles, provisioning users, and using tools like activity logs and Insights to monitor and optimize Docker usage. ## Who's this for? @@ -45,3 +37,162 @@ This guide covers integration with: - Okta - Entra ID SAML 2.0 - Azure Connect (OIDC) + +## Setting up roles and permissions in Docker + +With the right configurations, you can ensure your developers have easy access to necessary resources while preventing unauthorized access. This page guides you through identifying Docker users so you can allocate subscription seats efficiently within your Docker organization, and assigning roles to align with your organization's structure. + +### Identify your Docker users and accounts + +Before setting up roles and permissions, it's important to have a clear understanding of who in your organization requires Docker access. Focus on gathering a comprehensive view of active users, their roles within projects, and how they interact with Docker resources. This process can be supported by tools like device management software or manual assessments. Encourage all users to update their Docker accounts to use organizational email addresses, ensuring seamless integration with your subscription. + +For steps on how you can do this, see [step 1 of onboarding your organization](/manuals/admin/organization/setup/onboard.md). + +### Assign roles strategically + +When you invite members to join your Docker organization, you assign them a role. + +Docker's predefined roles offer flexibility for various organizational needs. Assigning roles effectively ensures a balance of accessibility and security. + +- Member: Non-administrative role. Members can view other members that are in the same organization. +- Editor: Partial administrative access to the organization. Editors can create, edit, and delete repositories. They can also edit an existing team's access permissions. +- Owner: Full organization administrative access. Owners can manage organization repositories, teams, members, settings, and billing. + +For more information, see [Roles and permissions](/manuals/enterprise/security/roles-and-permissions.md). + +#### Enhance with teams + +Teams in Docker provide a structured way to manage member access and they provide an additional level of permissions. They simplify permission management and enable consistent application of policies. + +- Organize users into teams aligned with projects, departments, or functional roles. This approach helps streamline resource allocation and ensures clarity in access control. +- Assign permissions at the team level rather than individually. For instance, a development team might have "Read & Write" access to certain repositories, while a QA team has "Read-only" access. +- As teams grow or responsibilities shift, you can easily update permissions or add new members, maintaining consistency without reconfiguring individual settings. + +For more information, see [Create and manage a team](/manuals/admin/organization/manage/manage-a-team.md). + +#### Example scenarios + +- Development teams: Assign the member role to developers, granting access to the repositories needed for coding and testing. +- Team leads: Assign the editor role to team leads for resource management and repository control within their teams. +- Organizational oversight: Restrict the organization owner or company owner roles to a select few trusted individuals responsible for billing and security settings. + +#### Best practices + +- Apply the principle of least privilege. Assign users only the minimum permissions necessary for their roles. +- Conduct regular reviews of role assignments to ensure they align with evolving team structures and organizational responsibilities. + +## Onboarding and managing roles and permissions in Docker + +This page guides you through onboarding owners and members, and using tools like SSO and SCIM to future-proof onboarding going forward. + +### Invite owners + +When you create a Docker organization, you automatically become its sole owner. While optional, adding additional owners can significantly ease the process of onboarding and managing your organization by distributing administrative responsibilities. It also ensures continuity and prevents blockers if the primary owner is unavailable. + +For detailed information on owners, see [Roles and permissions](/manuals/enterprise/security/roles-and-permissions.md). + +### Invite members and assign roles + +Members are granted controlled access to resources and enjoy enhanced organizational benefits. When you invite members to join your Docker organization, you immediately assign them a role. + +#### Benefits of inviting members + +- Enhanced visibility: Gain insights into user activity, making it easier to monitor access and enforce security policies. +- Streamlined collaboration: Help members collaborate effectively by granting access to shared resources and repositories. +- Improved resource management: Organize and track users within your organization, ensuring optimal allocation of resources. +- Access to enhanced features: Members benefit from organization-wide perks, such as increased pull limits and access to premium Docker features. +- Security control: Apply and enforce security settings at an organizational level, reducing risks associated with unmanaged accounts. + +For detailed information, see [Manage organization members](/manuals/admin/organization/manage/members.md). + +### Future-proof user management + +A robust, future-proof approach to user management combines automated provisioning, centralized authentication, and dynamic access control. Implementing these practices ensures a scalable, secure, and efficient environment. + +#### Secure user authentication with single sign-on (SSO) + +Integrating Docker with your identity provider streamlines user access and enhances security. + +SSO: + +- Simplifies sign in, as users sign in with their organizational credentials. +- Reduces password-related vulnerabilities. +- Simplifies onboarding as it works seamlessly with SCIM and group mapping for automated provisioning. + +For more information, see the [SSO documentation](/manuals/enterprise/security/single-sign-on/_index.md). + +#### Automate onboarding with SCIM and JIT provisioning + +Streamline user provisioning and role management with [SCIM](/manuals/enterprise/security/provisioning/scim/_index.md) and [Just-in-Time (JIT) provisioning](/manuals/enterprise/security/provisioning/just-in-time.md). + +With SCIM you can: + +- Sync users and roles automatically with your identity provider. +- Automate adding, updating, or removing users based on directory changes. + +With JIT provisioning you can: + +- Automatically add users upon first sign in based on [group mapping](#simplify-access-with-group-mapping). +- Reduce overhead by eliminating pre-invite steps. + +#### Simplify access with group mapping + +Group mapping automates permissions management by linking identity provider groups to Docker roles and teams. + +It also: + +- Reduces manual errors in role assignments. +- Ensures consistent access control policies. +- Help you scale permissions as teams grow or change. + +For more information on how it works, see [Group mapping](/manuals/enterprise/security/provisioning/scim/group-mapping.md). + +## Monitoring and insights + +Activity logs and Insights are useful tools for user and access management in Docker. They provide visibility into user actions, team workflows, and organizational trends, helping enhance security, ensure compliance, and boost productivity. + +### Activity logs + +Activity logs track events at the organization and repository levels, offering a clear view of activities like repository changes, team updates, and billing adjustments. + +Activity logs are available for Docker Team or Docker Business plans, with data retained for three months. + +#### Key features + +- Change tracking: View what changed, who made the change, and when. +- Comprehensive reporting: Monitor critical events such as repository creation, deletion, privacy changes, and role assignments. + +#### Example scenarios + +- Audit trail for security: A repository’s privacy settings were updated unexpectedly. The activity logs reveal which user made the change and when, helping administrators address potential security risks. +- Team collaboration review: Logs show which team members pushed updates to a critical repository, ensuring accountability during a development sprint. +- Billing adjustments: Track who added or removed subscription seats to maintain budgetary control and compliance. + +For more information, see [Activity logs](/manuals/admin/activity-logs.md). + +### Insights + +Insights provide data-driven views of Docker usage to improve team productivity and resource allocation. + +#### Key benefits + +- Standardized environments: Ensure consistent configurations and enforce best practices across teams. +- Improved visibility: Monitor metrics like Docker Desktop usage, builds, and container activity to understand team workflows and engagement. +- Optimized resources: Track license usage and feature adoption to maximize the value of your Docker subscription. + +#### Example scenarios + +- Usage trends: Identify underutilized licenses or resources, allowing reallocation to more active teams. +- Build efficiency: Track average build times and success rates to pinpoint bottlenecks in development processes. +- Container utilization: Analyze container activity across departments to ensure proper resource distribution and cost efficiency. + +For more information, see [Insights](/manuals/admin/insights.md). + +### Next steps + +Now that you've mastered user and access management in Docker, you can: + +- Review your [activity logs](/manuals/admin/activity-logs.md) regularly to maintain security awareness +- Check your [Insights dashboard](/manuals/admin/insights.md) to identify opportunities for optimization +- Explore [advanced security features](/manuals/enterprise/security/_index.md) to further enhance your Docker environment +- Share best practices with your team to ensure consistent adoption of security policies diff --git a/content/guides/admin-user-management/audit-and-monitor.md b/content/guides/admin-user-management/audit-and-monitor.md deleted file mode 100644 index 5fd0df91f1e2..000000000000 --- a/content/guides/admin-user-management/audit-and-monitor.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -title: Monitoring and insights -description: Track user actions, team workflows, and organizational trends with Activity logs and Insights to enhance security and productivity in Docker. -keywords: organizational insights, user management, access control, security, monitoring, admins -weight: 30 ---- - -Activity logs and Insights are useful tools for user and access management in Docker. They provide visibility into user actions, team workflows, and organizational trends, helping enhance security, ensure compliance, and boost productivity. - -## Activity logs - -Activity logs track events at the organization and repository levels, offering a clear view of activities like repository changes, team updates, and billing adjustments. - -Activity logs are available for Docker Team or Docker Business plans, with data retained for three months. - -### Key features - -- Change tracking: View what changed, who made the change, and when. -- Comprehensive reporting: Monitor critical events such as repository creation, deletion, privacy changes, and role assignments. - -### Example scenarios - -- Audit trail for security: A repository’s privacy settings were updated unexpectedly. The activity logs reveal which user made the change and when, helping administrators address potential security risks. -- Team collaboration review: Logs show which team members pushed updates to a critical repository, ensuring accountability during a development sprint. -- Billing adjustments: Track who added or removed subscription seats to maintain budgetary control and compliance. - -For more information, see [Activity logs](/manuals/admin/activity-logs.md). - -## Insights - -Insights provide data-driven views of Docker usage to improve team productivity and resource allocation. - -### Key benefits - -- Standardized environments: Ensure consistent configurations and enforce best practices across teams. -- Improved visibility: Monitor metrics like Docker Desktop usage, builds, and container activity to understand team workflows and engagement. -- Optimized resources: Track license usage and feature adoption to maximize the value of your Docker subscription. - -### Example scenarios - -- Usage trends: Identify underutilized licenses or resources, allowing reallocation to more active teams. -- Build efficiency: Track average build times and success rates to pinpoint bottlenecks in development processes. -- Container utilization: Analyze container activity across departments to ensure proper resource distribution and cost efficiency. - -For more information, see [Insights](/manuals/admin/insights.md). - -## Next steps - -Now that you've mastered user and access management in Docker, you can: - -- Review your [activity logs](/manuals/admin/activity-logs.md) regularly to maintain security awareness -- Check your [Insights dashboard](/manuals/admin/insights.md) to identify opportunities for optimization -- Explore [advanced security features](/manuals/enterprise/security/_index.md) to further enhance your Docker environment -- Share best practices with your team to ensure consistent adoption of security policies diff --git a/content/guides/admin-user-management/onboard.md b/content/guides/admin-user-management/onboard.md deleted file mode 100644 index a3d0c075b03b..000000000000 --- a/content/guides/admin-user-management/onboard.md +++ /dev/null @@ -1,70 +0,0 @@ ---- -title: Onboarding and managing roles and permissions in Docker -description: Learn how to manage roles, invite members, and implement scalable access control in Docker for secure and efficient collaboration. -keywords: sso, scim, jit, invite members, docker hub, docker admin console, onboarding, security -weight: 20 ---- - -This page guides you through onboarding owners and members, and using tools like SSO and SCIM to future-proof onboarding going forward. - -## Invite owners - -When you create a Docker organization, you automatically become its sole owner. While optional, adding additional owners can significantly ease the process of onboarding and managing your organization by distributing administrative responsibilities. It also ensures continuity and prevents blockers if the primary owner is unavailable. - -For detailed information on owners, see [Roles and permissions](/manuals/enterprise/security/roles-and-permissions.md). - -## Invite members and assign roles - -Members are granted controlled access to resources and enjoy enhanced organizational benefits. When you invite members to join your Docker organization, you immediately assign them a role. - -### Benefits of inviting members - -- Enhanced visibility: Gain insights into user activity, making it easier to monitor access and enforce security policies. -- Streamlined collaboration: Help members collaborate effectively by granting access to shared resources and repositories. -- Improved resource management: Organize and track users within your organization, ensuring optimal allocation of resources. -- Access to enhanced features: Members benefit from organization-wide perks, such as increased pull limits and access to premium Docker features. -- Security control: Apply and enforce security settings at an organizational level, reducing risks associated with unmanaged accounts. - -For detailed information, see [Manage organization members](/manuals/admin/organization/manage/members.md). - -## Future-proof user management - -A robust, future-proof approach to user management combines automated provisioning, centralized authentication, and dynamic access control. Implementing these practices ensures a scalable, secure, and efficient environment. - -### Secure user authentication with single sign-on (SSO) - -Integrating Docker with your identity provider streamlines user access and enhances security. - -SSO: - -- Simplifies sign in, as users sign in with their organizational credentials. -- Reduces password-related vulnerabilities. -- Simplifies onboarding as it works seamlessly with SCIM and group mapping for automated provisioning. - -For more information, see the [SSO documentation](/manuals/enterprise/security/single-sign-on/_index.md). - -### Automate onboarding with SCIM and JIT provisioning - -Streamline user provisioning and role management with [SCIM](/manuals/enterprise/security/provisioning/scim/_index.md) and [Just-in-Time (JIT) provisioning](/manuals/enterprise/security/provisioning/just-in-time.md). - -With SCIM you can: - -- Sync users and roles automatically with your identity provider. -- Automate adding, updating, or removing users based on directory changes. - -With JIT provisioning you can: - -- Automatically add users upon first sign in based on [group mapping](#simplify-access-with-group-mapping). -- Reduce overhead by eliminating pre-invite steps. - -### Simplify access with group mapping - -Group mapping automates permissions management by linking identity provider groups to Docker roles and teams. - -It also: - -- Reduces manual errors in role assignments. -- Ensures consistent access control policies. -- Help you scale permissions as teams grow or change. - -For more information on how it works, see [Group mapping](/manuals/enterprise/security/provisioning/scim/group-mapping.md). diff --git a/content/guides/admin-user-management/setup.md b/content/guides/admin-user-management/setup.md deleted file mode 100644 index 7bf82efeb714..000000000000 --- a/content/guides/admin-user-management/setup.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -title: Setting up roles and permissions in Docker -description: A guide to securely managing access and collaboration in Docker through roles and teams. -keywords: Docker roles, permissions management, access control, IT administration, team collaboration, least privilege, security, Docker Teams, role-based access -weight: 10 ---- - -With the right configurations, you can ensure your developers have easy access to necessary resources while preventing unauthorized access. This page guides you through identifying Docker users so you can allocate subscription seats efficiently within your Docker organization, and assigning roles to align with your organization's structure. - -## Identify your Docker users and accounts - -Before setting up roles and permissions, it's important to have a clear understanding of who in your organization requires Docker access. Focus on gathering a comprehensive view of active users, their roles within projects, and how they interact with Docker resources. This process can be supported by tools like device management software or manual assessments. Encourage all users to update their Docker accounts to use organizational email addresses, ensuring seamless integration with your subscription. - -For steps on how you can do this, see [step 1 of onboarding your organization](/manuals/admin/organization/setup/onboard.md). - -## Assign roles strategically - -When you invite members to join your Docker organization, you assign them a role. - -Docker's predefined roles offer flexibility for various organizational needs. Assigning roles effectively ensures a balance of accessibility and security. - -- Member: Non-administrative role. Members can view other members that are in the same organization. -- Editor: Partial administrative access to the organization. Editors can create, edit, and delete repositories. They can also edit an existing team's access permissions. -- Owner: Full organization administrative access. Owners can manage organization repositories, teams, members, settings, and billing. - -For more information, see [Roles and permissions](/manuals/enterprise/security/roles-and-permissions.md). - -### Enhance with teams - -Teams in Docker provide a structured way to manage member access and they provide an additional level of permissions. They simplify permission management and enable consistent application of policies. - -- Organize users into teams aligned with projects, departments, or functional roles. This approach helps streamline resource allocation and ensures clarity in access control. -- Assign permissions at the team level rather than individually. For instance, a development team might have "Read & Write" access to certain repositories, while a QA team has "Read-only" access. -- As teams grow or responsibilities shift, you can easily update permissions or add new members, maintaining consistency without reconfiguring individual settings. - -For more information, see [Create and manage a team](/manuals/admin/organization/manage/manage-a-team.md). - -### Example scenarios - -- Development teams: Assign the member role to developers, granting access to the repositories needed for coding and testing. -- Team leads: Assign the editor role to team leads for resource management and repository control within their teams. -- Organizational oversight: Restrict the organization owner or company owner roles to a select few trusted individuals responsible for billing and security settings. - -### Best practices - -- Apply the principle of least privilege. Assign users only the minimum permissions necessary for their roles. -- Conduct regular reviews of role assignments to ensure they align with evolving team structures and organizational responsibilities. diff --git a/content/guides/agentic-ai.md b/content/guides/agentic-ai.md index 9fb50bf5665e..26141201e775 100644 --- a/content/guides/agentic-ai.md +++ b/content/guides/agentic-ai.md @@ -5,7 +5,7 @@ keywords: AI, Docker, Model Runner, MCP Toolkit, AI agents, application developm summary: | Learn how to create AI agent applications using Docker Model Runner, and MCP Toolkit. params: - tags: [AI] + tags: [ai] time: 30 minutes --- diff --git a/content/guides/angular/_index.md b/content/guides/angular/_index.md index 6f28a0f99cf9..bd6c2fd1e59f 100644 --- a/content/guides/angular/_index.md +++ b/content/guides/angular/_index.md @@ -5,14 +5,18 @@ description: Containerize and develop Angular apps using Docker keywords: getting started, angular, docker, language, Dockerfile summary: | This guide explains how to containerize Angular applications using Docker. -toc_min: 1 -toc_max: 2 -languages: [js] +aliases: + - /guides/angular/configure-github-actions/ + - /guides/angular/containerize/ + - /guides/angular/deploy/ + - /guides/angular/develop/ + - /guides/angular/run-tests/ params: + tags: [cicd] time: 20 minutes - --- + The Angular language-specific guide shows you how to containerize an Angular application using Docker, following best practices for creating efficient, production-ready containers. [Angular](https://angular.dev/) is a robust and widely adopted framework for building dynamic, enterprise-grade web applications. However, managing dependencies, environments, and deployments can become complex as applications scale. Docker streamlines these challenges by offering a consistent, isolated environment for development and production. @@ -47,4 +51,1341 @@ Before you begin, ensure you have a working knowledge of: - Familiarity with [Angular](https://angular.io/) fundamentals. - Understanding of core Docker concepts such as images, containers, and Dockerfiles. If you're new to Docker, start with the [Docker basics](/get-started/docker-concepts/the-basics/what-is-a-container.md) guide. -Once you've completed the Angular getting started modules, you’ll be fully prepared to containerize your own Angular application using the detailed examples and best practices outlined in this guide. \ No newline at end of file +Once you've completed the Angular getting started modules, you’ll be fully prepared to containerize your own Angular application using the detailed examples and best practices outlined in this guide. + +## Containerize an Angular Application + +### Prerequisites + +Before you begin, make sure the following tools are installed and available on your system: + +- You have installed the latest version of [Docker Desktop](/get-started/get-docker.md). +- You have a [git client](https://git-scm.com/downloads). The examples in this section use a command-line based git client, but you can use any client. + +> **New to Docker?** +> Start with the [Docker basics](/get-started/docker-concepts/the-basics/what-is-a-container.md) guide to get familiar with key concepts like images, containers, and Dockerfiles. + +--- + +### Overview + +This guide walks you through the complete process of containerizing an Angular application with Docker. You’ll learn how to create a production-ready Docker image using best practices that improve performance, security, scalability, and deployment efficiency. + +By the end of this guide, you will: + +- Containerize an Angular application using Docker. +- Create and optimize a Dockerfile for production builds. +- Use multi-stage builds to minimize image size. +- Serve the application efficiently with a custom Nginx configuration. +- Build secure and maintainable Docker images by following best practices. + +--- + +### Get the sample application + +Clone the sample application to use with this guide. Open a terminal, navigate to the directory where you want to work, and run the following command +to clone the git repository: + +```console +$ git clone https://github.com/kristiyan-velkov/docker-angular-sample +``` +--- + +### Build the Docker image + +Angular is a front-end framework that compiles into static assets, so the Dockerfile uses a multi-stage build: one stage compiles the app with Node.js, and a second minimal stage serves the static output with Nginx. + +> [!TIP] +> +> [Gordon](/ai/gordon/), Docker's AI assistant, can generate Docker assets for your project. Ask Gordon to create a Dockerfile, Compose file, and `.dockerignore` tailored to your application. + +#### Step 1: Create the Dockerfile + +Before creating a Dockerfile, you need to choose a base image. You can either use the [Node.js Official Image](https://hub.docker.com/_/node) or a Docker Hardened Image (DHI) from the [Hardened Image catalog](https://hub.docker.com/hardened-images/catalog). + +Choosing DHI offers the advantage of a production-ready image that is lightweight and secure. For more information, see [Docker Hardened Images](https://docs.docker.com/dhi/). + +> [!IMPORTANT] +> This guide uses a stable Node.js LTS image tag that is considered secure when the guide is written. Because new releases and security patches are published regularly, the tag shown here may no longer be the safest option when you follow the guide. Always review the latest available image tags and select a secure, up-to-date version before building or deploying your application. +> +> Official Node.js Docker Images: https://hub.docker.com/_/node + +{{< tabs >}} +{{< tab name="Using Docker Hardened Images" >}} +Docker Hardened Images (DHIs) are available for Node.js in the [Docker Hardened Images catalog](https://hub.docker.com/hardened-images/catalog/dhi/node). Docker Hardened Images are freely available to everyone with no subscription required. You can pull and use them like any other Docker image after signing in to the DHI registry. For more information, see the [DHI quickstart](/dhi/get-started/) guide. + +1. Sign in to the DHI registry: + ```console + $ docker login dhi.io + ``` + +2. Pull the Node.js DHI (check the catalog for available versions): + ```console + $ docker pull dhi.io/node:24-alpine3.22-dev + ``` + +In the following Dockerfile, the `FROM` instruction uses `dhi.io/node:24-alpine3.22-dev` as the base image. + +```dockerfile +# ========================================= +# Stage 1: Build the Angular Application +# ========================================= + +# Use a lightweight DHI Node.js image for building +FROM dhi.io/node:24-alpine3.22-dev AS builder + +# Set the working directory inside the container +WORKDIR /app + +# Copy package-related files first to leverage Docker's caching mechanism +COPY package.json package-lock.json* ./ + +# Install project dependencies using npm ci (ensures a clean, reproducible install) +RUN --mount=type=cache,target=/root/.npm npm ci + +# Copy the rest of the application source code into the container +COPY . . + +# Build the Angular application +RUN npm run build + +# ========================================= +# Stage 2: Prepare Nginx to Serve Static Files +# ========================================= + +FROM dhi.io/nginx:1.28.0-alpine3.21-dev AS runner + +# Copy custom Nginx config +COPY nginx.conf /etc/nginx/nginx.conf + +# Copy the static build output from the build stage to Nginx's default HTML serving directory +COPY --chown=nginx:nginx --from=builder /app/dist/*/browser /usr/share/nginx/html + +# Use a non-root user for security best practices +USER nginx + +# Expose port 8080 to allow HTTP traffic +# Note: The default Nginx container now listens on port 8080 instead of 80 +EXPOSE 8080 + +# Start Nginx directly with custom config +ENTRYPOINT ["nginx", "-c", "/etc/nginx/nginx.conf"] +CMD ["-g", "daemon off;"] +``` + +{{< /tab >}} +{{< tab name="Using the Docker Official Image" >}} + +Create a file named `Dockerfile` with the following contents: + +```dockerfile +# ========================================= +# Stage 1: Build the Angular Application +# ========================================= +ARG NODE_VERSION=24.12.0-alpine +ARG NGINX_VERSION=alpine3.22 + +# Use a lightweight Node.js image for building (customizable via ARG) +FROM node:${NODE_VERSION} AS builder + +# Set the working directory inside the container +WORKDIR /app + +# Copy package-related files first to leverage Docker's caching mechanism +COPY package.json *package-lock.json* ./ + +# Install project dependencies using npm ci (ensures a clean, reproducible install) +RUN --mount=type=cache,target=/root/.npm npm ci + +# Copy the rest of the application source code into the container +COPY . . + +# Build the Angular application +RUN npm run build + +# ========================================= +# Stage 2: Prepare Nginx to Serve Static Files +# ========================================= + +FROM nginxinc/nginx-unprivileged:${NGINX_VERSION} AS runner + +# Copy custom Nginx config +COPY nginx.conf /etc/nginx/nginx.conf + +# Copy the static build output from the build stage to Nginx's default HTML serving directory +COPY --chown=nginx:nginx --from=builder /app/dist/*/browser /usr/share/nginx/html + +# Use a built-in non-root user for security best practices +USER nginx + +# Expose port 8080 to allow HTTP traffic +# Note: The default Nginx container now listens on port 8080 instead of 80 +EXPOSE 8080 + +# Start Nginx directly with custom config +ENTRYPOINT ["nginx", "-c", "/etc/nginx/nginx.conf"] +CMD ["-g", "daemon off;"] +``` + +> [!NOTE] +> We are using nginx-unprivileged instead of the standard Nginx image to follow security best practices. +> Running as a non-root user in the final image: +>- Reduces the attack surface +>- Aligns with Docker’s recommendations for container hardening +>- Helps comply with stricter security policies in production environments + +{{< /tab >}} +{{< /tabs >}} + +#### Step 2: Create the compose.yaml file + +Create a file named `compose.yaml` with the following contents: + +```yaml {collapse=true,title=compose.yaml} +services: + server: + build: + context: . + ports: + - 8080:8080 +``` + +#### Step 3: Create the .dockerignore file + +The `.dockerignore` file tells Docker which files and folders to exclude when building the image. + +> [!NOTE] +>This helps: +>- Reduce image size +>- Speed up the build process +>- Prevent sensitive or unnecessary files (like `.env`, `.git`, or `node_modules`) from being added to the final image. +> +> To learn more, visit the [.dockerignore reference](/reference/dockerfile.md#dockerignore-file). + +Create a file named `.dockerignore` with the following contents: + +```dockerignore +# ================================ +# Node and build output +# ================================ +node_modules +dist +out-tsc +.angular +.cache +.tmp + +# ================================ +# Testing & Coverage +# ================================ +coverage +jest +cypress +cypress/screenshots +cypress/videos +reports +playwright-report +.vite +.vitepress + +# ================================ +# Environment & log files +# ================================ +*.env* +!*.env.production +*.log +*.tsbuildinfo + +# ================================ +# IDE & OS-specific files +# ================================ +.vscode +.idea +.DS_Store +Thumbs.db +*.swp + +# ================================ +# Version control & CI files +# ================================ +.git +.gitignore + +# ================================ +# Docker & local orchestration +# ================================ +Dockerfile +Dockerfile.* +.dockerignore +docker-compose.yml +docker-compose*.yml + +# ================================ +# Miscellaneous +# ================================ +*.bak +*.old +*.tmp +``` + +#### Step 4: Create the `nginx.conf` file + +To serve your Angular application efficiently inside the container, you’ll configure Nginx with a custom setup. This configuration is optimized for performance, browser caching, gzip compression, and support for client-side routing. + +Create a file named `nginx.conf` in the root of your project directory, and add the following content: + +> [!NOTE] +> To learn more about configuring Nginx, see the [official Nginx documentation](https://nginx.org/en/docs/). + + +```nginx +worker_processes auto; + +pid /tmp/nginx.pid; + +events { + worker_connections 1024; +} + +http { + include /etc/nginx/mime.types; + default_type application/octet-stream; + + client_body_temp_path /tmp/client_temp; + proxy_temp_path /tmp/proxy_temp_path; + fastcgi_temp_path /tmp/fastcgi_temp; + uwsgi_temp_path /tmp/uwsgi_temp; + scgi_temp_path /tmp/scgi_temp; + + # Logging + access_log off; + error_log /dev/stderr warn; + + # Performance + sendfile on; + tcp_nopush on; + tcp_nodelay on; + keepalive_timeout 65; + keepalive_requests 1000; + + # Compression + gzip on; + gzip_vary on; + gzip_proxied any; + gzip_min_length 256; + gzip_comp_level 6; + gzip_types + text/plain + text/css + text/xml + text/javascript + application/javascript + application/x-javascript + application/json + application/xml + application/xml+rss + font/ttf + font/otf + image/svg+xml; + + server { + listen 8080; + server_name localhost; + + root /usr/share/nginx/html; + index index.html; + + # Angular Routing + location / { + try_files $uri $uri/ /index.html; + } + + # Static Assets Caching + location ~* \.(?:ico|css|js|gif|jpe?g|png|woff2?|eot|ttf|svg|map)$ { + expires 1y; + access_log off; + add_header Cache-Control "public, immutable"; + } + + # Optional: Explicit asset route + location /assets/ { + expires 1y; + add_header Cache-Control "public, immutable"; + } + } +} +``` + +#### Step 5: Build the Angular application image + +With your custom configuration in place, you're now ready to build the Docker image for your Angular application. + +The updated setup includes: + +- The updated setup includes a clean, production-ready Nginx configuration tailored specifically for Angular. +- Efficient multi-stage Docker build, ensuring a small and secure final image. + +After completing the previous steps, your project directory should now contain the following files: + +```text +├── docker-angular-sample/ +│ ├── Dockerfile +│ ├── .dockerignore +│ ├── compose.yaml +│ └── nginx.conf +``` + +Now that your Dockerfile is configured, you can build the Docker image for your Angular application. + +> [!NOTE] +> The `docker build` command packages your application into an image using the instructions in the Dockerfile. It includes all necessary files from the current directory (called the [build context](/build/concepts/context/#what-is-a-build-context)). + +Run the following command from the root of your project: + +```console +$ docker build --tag docker-angular-sample . +``` + +What this command does: +- Uses the Dockerfile in the current directory (.) +- Packages the application and its dependencies into a Docker image +- Tags the image as docker-angular-sample so you can reference it later + + +#### Step 6: View local images + +After building your Docker image, you can check which images are available on your local machine using either the Docker CLI or [Docker Desktop](/manuals/desktop/use-desktop/images.md). Since you're already working in the terminal, let's use the Docker CLI. + +To list all locally available Docker images, run the following command: + +```console +$ docker images +``` + +Example Output: + +```shell +REPOSITORY TAG IMAGE ID CREATED SIZE +docker-angular-sample latest 34e66bdb9d40 14 seconds ago 76.4MB +``` + +This output provides key details about your images: + +- **Repository** – The name assigned to the image. +- **Tag** – A version label that helps identify different builds (e.g., latest). +- **Image ID** – A unique identifier for the image. +- **Created** – The timestamp indicating when the image was built. +- **Size** – The total disk space used by the image. + +If the build was successful, you should see `docker-angular-sample` image listed. + +--- + +### Run the containerized application + +In the previous step, you created a Dockerfile for your Angular application and built a Docker image using the docker build command. Now it’s time to run that image in a container and verify that your application works as expected. + + +Inside the `docker-angular-sample` directory, run the following command in a +terminal. + +```console +$ docker compose up --build +``` + +Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see a simple Angular web application. + +Press `ctrl+c` in the terminal to stop your application. + +#### Run the application in the background + +You can run the application detached from the terminal by adding the `-d` +option. Inside the `docker-angular-sample` directory, run the following command +in a terminal. + +```console +$ docker compose up --build -d +``` + +Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see your Angular application running in the browser. + + +To confirm that the container is running, use `docker ps` command: + +```console +$ docker ps +``` + +This will list all active containers along with their ports, names, and status. Look for a container exposing port 8080. + +Example Output: + +```shell +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +eb13026806d1 docker-angular-sample-server "nginx -c /etc/nginx…" About a minute ago Up About a minute 0.0.0.0:8080->8080/tcp docker-angular-sample-server-1 +``` + + +To stop the application, run: + +```console +$ docker compose down +``` + + +> [!NOTE] +> For more information about Compose commands, see the [Compose CLI +> reference](/reference/cli/docker/compose/). + +--- + +### Summary + +In this guide, you learned how to containerize, build, and run an Angular application using Docker. By following best practices, you created a secure, optimized, and production-ready setup. + +What you accomplished: +- Created a multi-stage `Dockerfile` that compiles the Angular application and serves the static files using Nginx. +- Created a `.dockerignore` file to exclude unnecessary files and keep the image clean and efficient. +- Built your Docker image using `docker build`. +- Ran the container using `docker compose up`, both in the foreground and in detached mode. +- Verified that the app was running by visiting [http://localhost:8080](http://localhost:8080). +- Learned how to stop the containerized application using `docker compose down`. + +You now have a fully containerized Angular application, running in a Docker container, and ready for deployment across any environment with confidence and consistency. + +--- + +### Related resources + +Explore official references and best practices to sharpen your Docker workflow: + +- [Multi-stage builds](/build/building/multi-stage/) – Learn how to separate build and runtime stages. +- [Best practices for writing Dockerfiles](/develop/develop-images/dockerfile_best-practices/) – Write efficient, maintainable, and secure Dockerfiles. +- [Build context in Docker](/build/concepts/context/) – Learn how context affects image builds. +- [`docker build` CLI reference](/reference/cli/docker/image/build/) – Build Docker images from a Dockerfile. +- [`docker images` CLI reference](/reference/cli/docker/image/ls/) – Manage and inspect local Docker images. +- [`docker compose up` CLI reference](/reference/cli/docker/compose/up/) – Start and run multi-container applications. +- [`docker compose down` CLI reference](/reference/cli/docker/compose/down/) – Stop and remove containers, networks, and volumes. + +--- + +### Next steps + +With your Angular application now containerized, you're ready to move on to the next step. + +In the next section, you'll learn how to develop your application using Docker containers, enabling a consistent, isolated, and reproducible development environment across any machine. + +## Use containers for Angular development + +### Prerequisites + +Complete [Containerize Angular application](containerize.md). + +--- + +### Overview + +In this section, you'll learn how to set up both production and development environments for your containerized Angular application using Docker Compose. This setup allows you to serve a static production build via Nginx and to develop efficiently inside containers using a live-reloading dev server with Compose Watch. + +You’ll learn how to: +- Configure separate containers for production and development +- Enable automatic file syncing using Compose Watch in development +- Debug and live-preview your changes in real-time without manual rebuilds + +--- + +### Automatically update services (development mode) + +Use Compose Watch to automatically sync source file changes into your containerized development environment. This provides a seamless, efficient development experience without restarting or rebuilding containers manually. + +### Step 1: Create a development Dockerfile + +Create a file named `Dockerfile.dev` in your project root with the following content: + +```dockerfile +# ========================================= +# Stage 1: Development - Angular Application +# ========================================= + +# Define the Node.js version to use (Alpine for a small footprint) +ARG NODE_VERSION=24.12.0-alpine + +# Set the base image for development +FROM node:${NODE_VERSION} AS dev + +# Set environment variable to indicate development mode +ENV NODE_ENV=development + +# Set the working directory inside the container +WORKDIR /app + +# Copy only the dependency files first to optimize Docker caching +COPY package.json package-lock.json* ./ + +# Install dependencies using npm with caching to speed up subsequent builds +RUN --mount=type=cache,target=/root/.npm npm install + +# Copy all application source files into the container +COPY . . + +# Expose the port Angular uses for the dev server (default is 4200) +EXPOSE 4200 + +# Start the Angular dev server and bind it to all network interfaces +CMD ["npm", "start", "--", "--host=0.0.0.0"] + +``` + +This file sets up a lightweight development environment for your Angular application using the dev server. + + +#### Step 2: Update your `compose.yaml` file + +Open your `compose.yaml` file and define two services: one for production (`angular-prod`) and one for development (`angular-dev`). + +Here’s an example configuration for an Angular application: + +```yaml +services: + angular-prod: + build: + context: . + dockerfile: Dockerfile + image: docker-angular-sample + ports: + - "8080:8080" + + angular-dev: + build: + context: . + dockerfile: Dockerfile.dev + ports: + - "4200:4200" + develop: + watch: + - action: sync + path: . + target: /app +``` +- The `angular-prod` service builds and serves your static production app using Nginx. +- The `angular-dev` service runs your Angular development server with live reload and hot module replacement. +- `watch` triggers file sync with Compose Watch. + +> [!NOTE] +> For more details, see the official guide: [Use Compose Watch](/manuals/compose/how-tos/file-watch.md). + +After completing the previous steps, your project directory should now contain the following files: + +```text +├── docker-angular-sample/ +│ ├── Dockerfile +│ ├── Dockerfile.dev +│ ├── .dockerignore +│ ├── compose.yaml +│ └── nginx.conf +``` + +#### Step 4: Start Compose Watch + +Run the following command from the project root to start the container in watch mode + +```console +$ docker compose watch angular-dev +``` + +#### Step 5: Test Compose Watch with Angular + +To verify that Compose Watch is working correctly: + +1. Open the `src/app/app.component.html` file in your text editor. + +2. Locate the following line: + + ```html +

Docker Angular Sample Application

+ ``` + +3. Change it to: + + ```html +

Hello from Docker Compose Watch

+ ``` + +4. Save the file. + +5. Open your browser at [http://localhost:4200](http://localhost:4200). + +You should see the updated text appear instantly, without needing to rebuild the container manually. This confirms that file watching and automatic synchronization are working as expected. + +--- + +### Summary + +In this section, you set up a complete development and production workflow for your Angular application using Docker and Docker Compose. + +Here’s what you accomplished: +- Created a `Dockerfile.dev` to streamline local development with hot reloading +- Defined separate `angular-dev` and `angular-prod` services in your `compose.yaml` file +- Enabled real-time file syncing using Compose Watch for a smoother development experience +- Verified that live updates work seamlessly by modifying and previewing a component + +With this setup, you're now equipped to build, run, and iterate on your Angular app entirely within containers—efficiently and consistently across environments. + +--- + +### Related resources + +Deepen your knowledge and improve your containerized development workflow with these guides: + +- [Using Compose Watch](/manuals/compose/how-tos/file-watch.md) – Automatically sync source changes during development +- [Multi-stage builds](/manuals/build/building/multi-stage.md) – Create efficient, production-ready Docker images +- [Dockerfile best practices](/build/building/best-practices/) – Write clean, secure, and optimized Dockerfiles. +- [Compose file reference](/compose/compose-file/) – Learn the full syntax and options available for configuring services in `compose.yaml`. +- [Docker volumes](/storage/volumes/) – Persist and manage data between container runs + +### Next steps + +In the next section, you'll learn how to run unit tests for your Angular application inside Docker containers. This ensures consistent testing across all environments and removes dependencies on local machine setup. + +## Run Angular tests in a container + +### Prerequisites + +Complete all the previous sections of this guide, starting with [Containerize Angular application](containerize.md). + +### Overview + +Testing is a critical part of the development process. In this section, you'll learn how to: + +- Run Jasmine unit tests using the Angular CLI inside a Docker container. +- Use Docker Compose to isolate your test environment. +- Ensure consistency between local and container-based testing. + + +The `docker-angular-sample` project comes pre-configured with Jasmine, so you can get started quickly without extra setup. + +--- + +### Run tests during development + +The `docker-angular-sample` application includes a sample test file at the following location: + +```console +$ src/app/app.component.spec.ts +``` + +This test uses Jasmine to validate the AppComponent logic. + +#### Step 1: Update compose.yaml + +Add a new service named `angular-test` to your `compose.yaml` file. This service allows you to run your test suite in an isolated, containerized environment. + +```yaml {hl_lines="22-26",linenos=true} +services: + angular-dev: + build: + context: . + dockerfile: Dockerfile.dev + ports: + - "5173:5173" + develop: + watch: + - action: sync + path: . + target: /app + + angular-prod: + build: + context: . + dockerfile: Dockerfile + image: docker-angular-sample + ports: + - "8080:8080" + + angular-test: + build: + context: . + dockerfile: Dockerfile.dev + command: ["npm", "run", "test"] + +``` + +The angular-test service reuses the same `Dockerfile.dev` used for [development](develop.md) and overrides the default command to run tests with `npm run test`. This setup ensures a consistent test environment that matches your local development configuration. + + +After completing the previous steps, your project directory should contain the following files: + +```text +├── docker-angular-sample/ +│ ├── Dockerfile +│ ├── Dockerfile.dev +│ ├── .dockerignore +│ ├── compose.yaml +│ └── nginx.conf +``` + +#### Step 2: Run the tests + +To execute your test suite inside the container, run the following command from your project root: + +```console +$ docker compose run --rm angular-test +``` + +This command will: +- Start the `angular-test` service defined in your `compose.yaml` file. +- Execute the `npm run test` script using the same environment as development. +- Automatically removes the container after tests complete, using the [`docker compose run --rm`](/reference/cli/docker/compose/run/) command. + +You should see output similar to the following: + +```shell +Test Suites: 1 passed, 1 total +Tests: 3 passed, 3 total +Snapshots: 0 total +Time: 1.529 s +``` + +> [!NOTE] +> For more information about Compose commands, see the [Compose CLI +> reference](/reference/cli/docker/compose/). + +--- + +### Summary + +In this section, you learned how to run unit tests for your Angular application inside a Docker container using Jasmine and Docker Compose. + +What you accomplished: +- Created a `angular-test` service in `compose.yaml` to isolate test execution. +- Reused the development `Dockerfile.dev` to ensure consistency between dev and test environments. +- Ran tests inside the container using `docker compose run --rm angular-test`. +- Ensured reliable, repeatable testing across environments without depending on your local machine setup. + +--- + +### Related resources + +Explore official references and best practices to sharpen your Docker testing workflow: + +- [Dockerfile reference](/reference/dockerfile/) – Understand all Dockerfile instructions and syntax. +- [Best practices for writing Dockerfiles](/develop/develop-images/dockerfile_best-practices/) – Write efficient, maintainable, and secure Dockerfiles. +- [Compose file reference](/compose/compose-file/) – Learn the full syntax and options available for configuring services in `compose.yaml`. +- [`docker compose run` CLI reference](/reference/cli/docker/compose/run/) – Run one-off commands in a service container. +--- + +### Next steps + +Next, you’ll learn how to set up a CI/CD pipeline using GitHub Actions to automatically build and test your Angular application in a containerized environment. This ensures your code is validated on every push or pull request, maintaining consistency and reliability across your development workflow. + +## Test your Angular deployment + +### Prerequisites + +Before you begin, make sure you’ve completed the following: +- Complete all the previous sections of this guide, starting with [Containerize Angular application](containerize.md). +- [Enable Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. + +> **New to Kubernetes?** +> Visit the [Kubernetes basics tutorial](https://kubernetes.io/docs/tutorials/kubernetes-basics/) to get familiar with how clusters, pods, deployments, and services work. + +--- + +### Overview + +This section guides you through deploying your containerized Angular application locally using [Docker Desktop’s built-in Kubernetes](/desktop/kubernetes/). Running your app in a local Kubernetes cluster closely simulates a real production environment, enabling you to test, validate, and debug your workloads with confidence before promoting them to staging or production. + +--- + +### Create a Kubernetes YAML file + +Follow these steps to define your deployment configuration: + +1. In the root of your project, create a new file named: angular-sample-kubernetes.yaml + +2. Open the file in your IDE or preferred text editor. + +3. Add the following configuration, and be sure to replace `{DOCKER_USERNAME}` and `{DOCKERHUB_PROJECT_NAME}` with your actual Docker Hub username and repository name from the previous [Automate your builds with GitHub Actions](configure-github-actions.md). + + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: angular-sample + namespace: default +spec: + replicas: 1 + selector: + matchLabels: + app: angular-sample + template: + metadata: + labels: + app: angular-sample + spec: + containers: + - name: angular-container + image: {DOCKER_USERNAME}/{DOCKERHUB_PROJECT_NAME}:latest + imagePullPolicy: Always + ports: + - containerPort: 8080 + resources: + limits: + cpu: "500m" + memory: "256Mi" + requests: + cpu: "250m" + memory: "128Mi" +--- +apiVersion: v1 +kind: Service +metadata: + name: angular-sample-service + namespace: default +spec: + type: NodePort + selector: + app: angular-sample + ports: + - port: 8080 + targetPort: 8080 + nodePort: 30001 +``` + +This manifest defines two key Kubernetes resources, separated by `---`: + +- Deployment + Deploys a single replica of your Angular application inside a pod. The pod uses the Docker image built and pushed by your GitHub Actions CI/CD workflow + (refer to [Automate your builds with GitHub Actions](configure-github-actions.md)). + The container listens on port `8080`, which is typically used by [Nginx](https://nginx.org/en/docs/) to serve your production Angular app. + +- Service (NodePort) + Exposes the deployed pod to your local machine. + It forwards traffic from port `30001` on your host to port `8080` inside the container. + This lets you access the application in your browser at [http://localhost:30001](http://localhost:30001). + +> [!NOTE] +> To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). + +--- + +### Deploy and check your application + +Follow these steps to deploy your containerized Angular app into a local Kubernetes cluster and verify that it’s running correctly. + +#### Step 1. Apply the Kubernetes configuration + +In your terminal, navigate to the directory where your `angular-sample-kubernetes.yaml` file is located, then deploy the resources using: + +```console + $ kubectl apply -f angular-sample-kubernetes.yaml +``` + +If everything is configured properly, you’ll see confirmation that both the Deployment and the Service were created: + +```shell + deployment.apps/angular-sample created + service/angular-sample-service created +``` + +This confirms that both the Deployment and the Service were successfully created and are now running inside your local cluster. + +#### Step 2. Check the deployment status + +Run the following command to check the status of your deployment: + +```console + $ kubectl get deployments +``` + +You should see output similar to the following: + +```shell + NAME READY UP-TO-DATE AVAILABLE AGE + angular-sample 1/1 1 1 14s +``` + +This confirms that your pod is up and running with one replica available. + +#### Step 3. Verify the service exposure + +Check if the NodePort service is exposing your app to your local machine: + +```console +$ kubectl get services +``` + +You should see something like: + +```shell +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +angular-sample-service NodePort 10.100.185.105 8080:30001/TCP 1m +``` + +This output confirms that your app is available via NodePort on port 30001. + +#### Step 4. Access your app in the browser + +Open your browser and navigate to [http://localhost:30001](http://localhost:30001). + +You should see your production-ready Angular Sample application running — served by your local Kubernetes cluster. + +#### Step 5. Clean up Kubernetes resources + +Once you're done testing, you can delete the deployment and service using: + +```console + $ kubectl delete -f angular-sample-kubernetes.yaml +``` + +Expected output: + +```shell + deployment.apps "angular-sample" deleted + service "angular-sample-service" deleted +``` + +This ensures your cluster stays clean and ready for the next deployment. + +--- + +### Summary + +In this section, you learned how to deploy your Angular application to a local Kubernetes cluster using Docker Desktop. This setup allows you to test and debug your containerized app in a production-like environment before deploying it to the cloud. + +What you accomplished: + +- Created a Kubernetes Deployment and NodePort Service for your Angular app +- Used `kubectl apply` to deploy the application locally +- Verified the app was running and accessible at `http://localhost:30001` +- Cleaned up your Kubernetes resources after testing + +--- + +### Related resources + +Explore official references and best practices to sharpen your Kubernetes deployment workflow: + +- [Kubernetes documentation](https://kubernetes.io/docs/home/) – Learn about core concepts, workloads, services, and more. +- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) – Use Docker Desktop’s built-in Kubernetes support for local testing and development. +- [`kubectl` CLI reference](https://kubernetes.io/docs/reference/kubectl/) – Manage Kubernetes clusters from the command line. +- [Kubernetes Deployment resource](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) – Understand how to manage and scale applications using Deployments. +- [Kubernetes Service resource](https://kubernetes.io/docs/concepts/services-networking/service/) – Learn how to expose your application to internal and external traffic. + +## Automate your builds with GitHub Actions + +### Prerequisites + +Complete all the previous sections of this guide, starting with [Containerize an Angular application](containerize.md). + +You must also have: +- A [GitHub](https://github.com/signup) account. +- A verified [Docker Hub](https://hub.docker.com/signup) account. + +--- + +### Overview + +In this section, you'll set up a CI/CD pipeline using [GitHub Actions](https://docs.github.com/en/actions) to automatically: + +- Build your Angular application inside a Docker container. +- Run tests in a consistent environment. +- Push the production-ready image to [Docker Hub](https://hub.docker.com). + +--- + +### Connect your GitHub repository to Docker Hub + +To enable GitHub Actions to build and push Docker images, you’ll securely store your Docker Hub credentials in your new GitHub repository. + +#### Step 1: Generate Docker Hub credentials and set GitHub secrets + +1. Create a Personal Access Token (PAT) from [Docker Hub](https://hub.docker.com) + 1. Go to your **Docker Hub account → Account Settings → Security**. + 2. Generate a new Access Token with **Read/Write** permissions. + 3. Name it something like `docker-angular-sample`. + 4. Copy and save the token — you’ll need it in Step 4. + +2. Create a repository in [Docker Hub](https://hub.docker.com/repositories/) + 1. Go to your **Docker Hub account → Create a repository**. + 2. For the Repository Name, use something descriptive — for example: `angular-sample`. + 3. Once created, copy and save the repository name — you’ll need it in Step 4. + +3. Create a new [GitHub repository](https://github.com/new) for your Angular project + +4. Add Docker Hub credentials as GitHub repository secrets + + In your newly created GitHub repository: + + 1. Navigate to: + **Settings → Secrets and variables → Actions → New repository secret**. + + 2. Add the following secrets: + + | Name | Value | + |-------------------|--------------------------------| + | `DOCKER_USERNAME` | Your Docker Hub username | + | `DOCKERHUB_TOKEN` | Your Docker Hub access token (created in Step 1) | + | `DOCKERHUB_PROJECT_NAME` | Your Docker Project Name (created in Step 2) | + + These secrets allow GitHub Actions to authenticate securely with Docker Hub during automated workflows. + +5. Connect Your Local Project to GitHub + + Link your local project `docker-angular-sample` to the GitHub repository you just created by running the following command from your project root: + + ```console + $ git remote set-url origin https://github.com/{your-username}/{your-repository-name}.git + ``` + + >[!IMPORTANT] + >Replace `{your-username}` and `{your-repository}` with your actual GitHub username and repository name. + + To confirm that your local project is correctly connected to the remote GitHub repository, run: + + ```console + $ git remote -v + ``` + + You should see output similar to: + + ```console + origin https://github.com/{your-username}/{your-repository-name}.git (fetch) + origin https://github.com/{your-username}/{your-repository-name}.git (push) + ``` + + This confirms that your local repository is properly linked and ready to push your source code to GitHub. + +6. Push your source code to GitHub + + Follow these steps to commit and push your local project to your GitHub repository: + + 1. Stage all files for commit. + + ```console + $ git add -A + ``` + This command stages all changes — including new, modified, and deleted files — preparing them for commit. + + + 2. Commit the staged changes with a descriptive message. + + ```console + $ git commit -m "Initial commit" + ``` + This command creates a commit that snapshots the staged changes with a descriptive message. + + 3. Push the code to the `main` branch. + + ```console + $ git push -u origin main + ``` + This command pushes your local commits to the `main` branch of the remote GitHub repository and sets the upstream branch. + +Once completed, your code will be available on GitHub, and any GitHub Actions workflow you’ve configured will run automatically. + +> [!NOTE] +> Learn more about the Git commands used in this step: +> - [Git add](https://git-scm.com/docs/git-add) – Stage changes (new, modified, deleted) for commit +> - [Git commit](https://git-scm.com/docs/git-commit) – Save a snapshot of your staged changes +> - [Git push](https://git-scm.com/docs/git-push) – Upload local commits to your GitHub repository +> - [Git remote](https://git-scm.com/docs/git-remote) – View and manage remote repository URLs + +--- + +#### Step 2: Set up the workflow + +Now you'll create a GitHub Actions workflow that builds your Docker image, runs tests, and pushes the image to Docker Hub. + +1. Go to your repository on GitHub and select the **Actions** tab in the top menu. + +2. Select **Set up a workflow yourself** when prompted. + + This opens an inline editor to create a new workflow file. By default, it will be saved to: + `.github/workflows/main.yml` + + +3. Add the following workflow configuration to the new file: + +```yaml +name: CI/CD – Angular Application with Docker + +on: + push: + branches: [main] + pull_request: + branches: [main] + types: [opened, synchronize, reopened] + +jobs: + build-test-push: + name: Build, Test, and Push Docker Image + runs-on: ubuntu-latest + + steps: + # 1. Checkout source code + - name: Checkout source code + uses: actions/checkout@{{% param "checkout_action_version" %}} + with: + fetch-depth: 0 + + # 2. Set up Docker Buildx + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} + + # 3. Cache Docker layers + - name: Cache Docker layers + uses: actions/cache@{{% param "cache_action_version" %}} + with: + path: /tmp/.buildx-cache + key: ${{ runner.os }}-buildx-${{ github.sha }} + restore-keys: | + ${{ runner.os }}-buildx- + + # 4. Cache npm dependencies + - name: Cache npm dependencies + uses: actions/cache@{{% param "cache_action_version" %}} + with: + path: ~/.npm + key: ${{ runner.os }}-npm-${{ hashFiles('**/package-lock.json') }} + restore-keys: | + ${{ runner.os }}-npm- + + # 5. Extract metadata + - name: Extract metadata + id: meta + run: | + echo "REPO_NAME=${GITHUB_REPOSITORY##*/}" >> "$GITHUB_OUTPUT" + echo "SHORT_SHA=${GITHUB_SHA::7}" >> "$GITHUB_OUTPUT" + + # 6. Build dev Docker image + - name: Build Docker image for tests + uses: docker/build-push-action@{{% param "build_push_action_version" %}} + with: + context: . + file: Dockerfile.dev + tags: ${{ steps.meta.outputs.REPO_NAME }}-dev:latest + load: true + cache-from: type=local,src=/tmp/.buildx-cache + cache-to: type=local,dest=/tmp/.buildx-cache,mode=max + + # 7. Run Angular tests with Jasmine + - name: Run Angular Jasmine tests inside container + run: | + docker run --rm \ + --workdir /app \ + --entrypoint "" \ + ${{ steps.meta.outputs.REPO_NAME }}-dev:latest \ + sh -c "npm ci && npm run test -- --ci --runInBand" + env: + CI: true + NODE_ENV: test + timeout-minutes: 10 + + # 8. Log in to Docker Hub + - name: Log in to Docker Hub + uses: docker/login-action@{{% param "login_action_version" %}} + with: + username: ${{ secrets.DOCKER_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + + # 9. Build and push production image + - name: Build and push production image + uses: docker/build-push-action@{{% param "build_push_action_version" %}} + with: + context: . + file: Dockerfile + push: true + platforms: linux/amd64,linux/arm64 + tags: | + ${{ secrets.DOCKER_USERNAME }}/${{ secrets.DOCKERHUB_PROJECT_NAME }}:latest + ${{ secrets.DOCKER_USERNAME }}/${{ secrets.DOCKERHUB_PROJECT_NAME }}:${{ steps.meta.outputs.SHORT_SHA }} + cache-from: type=local,src=/tmp/.buildx-cache +``` + +This workflow performs the following tasks for your Angular application: +- Triggers on every `push` or `pull request` targeting the `main` branch. +- Builds a development Docker image using `Dockerfile.dev`, optimized for testing. +- Executes unit tests using Vitest inside a clean, containerized environment to ensure consistency. +- Halts the workflow immediately if any test fails — enforcing code quality. +- Caches both Docker build layers and npm dependencies for faster CI runs. +- Authenticates securely with Docker Hub using GitHub repository secrets. +- Builds a production-ready image using the `prod` stage in `Dockerfile`. +- Tags and pushes the final image to Docker Hub with both `latest` and short SHA tags for traceability. + +> [!NOTE] +> For more information about `docker/build-push-action`, refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). + +--- + +#### Step 3: Run the workflow + +After you've added your workflow file, it's time to trigger and observe the CI/CD process in action. + +1. Commit and push your workflow file + + - Select "Commit changes…" in the GitHub editor. + + - This push will automatically trigger the GitHub Actions pipeline. + +2. Monitor the workflow execution + + - Go to the Actions tab in your GitHub repository. + - Click into the workflow run to follow each step: **build**, **test**, and (if successful) **push**. + +3. Verify the Docker image on Docker Hub + + - After a successful workflow run, visit your [Docker Hub repositories](https://hub.docker.com/repositories). + - You should see a new image under your repository with: + - Repository name: `${your-repository-name}` + - Tags include: + - `latest` – represents the most recent successful build; ideal for quick testing or deployment. + - `` – a unique identifier based on the commit hash, useful for version tracking, rollbacks, and traceability. + +> [!TIP] Protect your main branch +> To maintain code quality and prevent accidental direct pushes, enable branch protection rules: +> - Navigate to your **GitHub repo → Settings → Branches**. +> - Under Branch protection rules, click **Add rule**. +> - Specify `main` as the branch name. +> - Enable options like: +> - *Require a pull request before merging*. +> - *Require status checks to pass before merging*. +> +> This ensures that only tested and reviewed code is merged into `main` branch. +--- + +### Summary + +In this section, you set up a complete CI/CD pipeline for your containerized Angular application using GitHub Actions. + +Here's what you accomplished: + +- Created a new GitHub repository specifically for your project. +- Generated a secure Docker Hub access token and added it to GitHub as a secret. +- Defined a GitHub Actions workflow that: + - Build your application inside a Docker container. + - Run tests in a consistent, containerized environment. + - Push a production-ready image to Docker Hub if tests pass. +- Triggered and verified the workflow execution through GitHub Actions. +- Confirmed that your image was successfully published to Docker Hub. + +With this setup, your Angular application is now ready for automated testing and deployment across environments — increasing confidence, consistency, and team productivity. + +--- + +### Related resources + +Deepen your understanding of automation and best practices for containerized apps: + +- [Introduction to GitHub Actions](/guides/gha.md) – Learn how GitHub Actions automate your workflows +- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) – Set up container builds with GitHub Actions +- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) – Full reference for writing GitHub workflows +- [Compose file reference](/compose/compose-file/) – Full configuration reference for `compose.yaml` +- [Best practices for writing Dockerfiles](/develop/develop-images/dockerfile_best-practices/) – Optimize your image for performance and security + +--- + +### Next steps + +Next, learn how you can locally test and debug your Angular workloads on Kubernetes before deploying. This helps you ensure your application behaves as expected in a production-like environment, reducing surprises during deployment. diff --git a/content/guides/angular/configure-github-actions.md b/content/guides/angular/configure-github-actions.md deleted file mode 100644 index 335ea0f304d5..000000000000 --- a/content/guides/angular/configure-github-actions.md +++ /dev/null @@ -1,323 +0,0 @@ ---- -title: Automate your builds with GitHub Actions -linkTitle: GitHub Actions CI -weight: 60 -keywords: CI/CD, GitHub( Actions), Angular -description: Learn how to configure CI/CD using GitHub Actions for your Angular application. - ---- - -## Prerequisites - -Complete all the previous sections of this guide, starting with [Containerize an Angular application](containerize.md). - -You must also have: -- A [GitHub](https://github.com/signup) account. -- A verified [Docker Hub](https://hub.docker.com/signup) account. - ---- - -## Overview - -In this section, you'll set up a CI/CD pipeline using [GitHub Actions](https://docs.github.com/en/actions) to automatically: - -- Build your Angular application inside a Docker container. -- Run tests in a consistent environment. -- Push the production-ready image to [Docker Hub](https://hub.docker.com). - ---- - -## Connect your GitHub repository to Docker Hub - -To enable GitHub Actions to build and push Docker images, you’ll securely store your Docker Hub credentials in your new GitHub repository. - -### Step 1: Generate Docker Hub credentials and set GitHub secrets - -1. Create a Personal Access Token (PAT) from [Docker Hub](https://hub.docker.com) - 1. Go to your **Docker Hub account → Account Settings → Security**. - 2. Generate a new Access Token with **Read/Write** permissions. - 3. Name it something like `docker-angular-sample`. - 4. Copy and save the token — you’ll need it in Step 4. - -2. Create a repository in [Docker Hub](https://hub.docker.com/repositories/) - 1. Go to your **Docker Hub account → Create a repository**. - 2. For the Repository Name, use something descriptive — for example: `angular-sample`. - 3. Once created, copy and save the repository name — you’ll need it in Step 4. - -3. Create a new [GitHub repository](https://github.com/new) for your Angular project - -4. Add Docker Hub credentials as GitHub repository secrets - - In your newly created GitHub repository: - - 1. Navigate to: - **Settings → Secrets and variables → Actions → New repository secret**. - - 2. Add the following secrets: - - | Name | Value | - |-------------------|--------------------------------| - | `DOCKER_USERNAME` | Your Docker Hub username | - | `DOCKERHUB_TOKEN` | Your Docker Hub access token (created in Step 1) | - | `DOCKERHUB_PROJECT_NAME` | Your Docker Project Name (created in Step 2) | - - These secrets allow GitHub Actions to authenticate securely with Docker Hub during automated workflows. - -5. Connect Your Local Project to GitHub - - Link your local project `docker-angular-sample` to the GitHub repository you just created by running the following command from your project root: - - ```console - $ git remote set-url origin https://github.com/{your-username}/{your-repository-name}.git - ``` - - >[!IMPORTANT] - >Replace `{your-username}` and `{your-repository}` with your actual GitHub username and repository name. - - To confirm that your local project is correctly connected to the remote GitHub repository, run: - - ```console - $ git remote -v - ``` - - You should see output similar to: - - ```console - origin https://github.com/{your-username}/{your-repository-name}.git (fetch) - origin https://github.com/{your-username}/{your-repository-name}.git (push) - ``` - - This confirms that your local repository is properly linked and ready to push your source code to GitHub. - -6. Push your source code to GitHub - - Follow these steps to commit and push your local project to your GitHub repository: - - 1. Stage all files for commit. - - ```console - $ git add -A - ``` - This command stages all changes — including new, modified, and deleted files — preparing them for commit. - - - 2. Commit the staged changes with a descriptive message. - - ```console - $ git commit -m "Initial commit" - ``` - This command creates a commit that snapshots the staged changes with a descriptive message. - - 3. Push the code to the `main` branch. - - ```console - $ git push -u origin main - ``` - This command pushes your local commits to the `main` branch of the remote GitHub repository and sets the upstream branch. - -Once completed, your code will be available on GitHub, and any GitHub Actions workflow you’ve configured will run automatically. - -> [!NOTE] -> Learn more about the Git commands used in this step: -> - [Git add](https://git-scm.com/docs/git-add) – Stage changes (new, modified, deleted) for commit -> - [Git commit](https://git-scm.com/docs/git-commit) – Save a snapshot of your staged changes -> - [Git push](https://git-scm.com/docs/git-push) – Upload local commits to your GitHub repository -> - [Git remote](https://git-scm.com/docs/git-remote) – View and manage remote repository URLs - ---- - -### Step 2: Set up the workflow - -Now you'll create a GitHub Actions workflow that builds your Docker image, runs tests, and pushes the image to Docker Hub. - -1. Go to your repository on GitHub and select the **Actions** tab in the top menu. - -2. Select **Set up a workflow yourself** when prompted. - - This opens an inline editor to create a new workflow file. By default, it will be saved to: - `.github/workflows/main.yml` - - -3. Add the following workflow configuration to the new file: - -```yaml -name: CI/CD – Angular Application with Docker - -on: - push: - branches: [main] - pull_request: - branches: [main] - types: [opened, synchronize, reopened] - -jobs: - build-test-push: - name: Build, Test, and Push Docker Image - runs-on: ubuntu-latest - - steps: - # 1. Checkout source code - - name: Checkout source code - uses: actions/checkout@{{% param "checkout_action_version" %}} - with: - fetch-depth: 0 - - # 2. Set up Docker Buildx - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} - - # 3. Cache Docker layers - - name: Cache Docker layers - uses: actions/cache@{{% param "cache_action_version" %}} - with: - path: /tmp/.buildx-cache - key: ${{ runner.os }}-buildx-${{ github.sha }} - restore-keys: | - ${{ runner.os }}-buildx- - - # 4. Cache npm dependencies - - name: Cache npm dependencies - uses: actions/cache@{{% param "cache_action_version" %}} - with: - path: ~/.npm - key: ${{ runner.os }}-npm-${{ hashFiles('**/package-lock.json') }} - restore-keys: | - ${{ runner.os }}-npm- - - # 5. Extract metadata - - name: Extract metadata - id: meta - run: | - echo "REPO_NAME=${GITHUB_REPOSITORY##*/}" >> "$GITHUB_OUTPUT" - echo "SHORT_SHA=${GITHUB_SHA::7}" >> "$GITHUB_OUTPUT" - - # 6. Build dev Docker image - - name: Build Docker image for tests - uses: docker/build-push-action@{{% param "build_push_action_version" %}} - with: - context: . - file: Dockerfile.dev - tags: ${{ steps.meta.outputs.REPO_NAME }}-dev:latest - load: true - cache-from: type=local,src=/tmp/.buildx-cache - cache-to: type=local,dest=/tmp/.buildx-cache,mode=max - - # 7. Run Angular tests with Jasmine - - name: Run Angular Jasmine tests inside container - run: | - docker run --rm \ - --workdir /app \ - --entrypoint "" \ - ${{ steps.meta.outputs.REPO_NAME }}-dev:latest \ - sh -c "npm ci && npm run test -- --ci --runInBand" - env: - CI: true - NODE_ENV: test - timeout-minutes: 10 - - # 8. Log in to Docker Hub - - name: Log in to Docker Hub - uses: docker/login-action@{{% param "login_action_version" %}} - with: - username: ${{ secrets.DOCKER_USERNAME }} - password: ${{ secrets.DOCKERHUB_TOKEN }} - - # 9. Build and push production image - - name: Build and push production image - uses: docker/build-push-action@{{% param "build_push_action_version" %}} - with: - context: . - file: Dockerfile - push: true - platforms: linux/amd64,linux/arm64 - tags: | - ${{ secrets.DOCKER_USERNAME }}/${{ secrets.DOCKERHUB_PROJECT_NAME }}:latest - ${{ secrets.DOCKER_USERNAME }}/${{ secrets.DOCKERHUB_PROJECT_NAME }}:${{ steps.meta.outputs.SHORT_SHA }} - cache-from: type=local,src=/tmp/.buildx-cache -``` - -This workflow performs the following tasks for your Angular application: -- Triggers on every `push` or `pull request` targeting the `main` branch. -- Builds a development Docker image using `Dockerfile.dev`, optimized for testing. -- Executes unit tests using Vitest inside a clean, containerized environment to ensure consistency. -- Halts the workflow immediately if any test fails — enforcing code quality. -- Caches both Docker build layers and npm dependencies for faster CI runs. -- Authenticates securely with Docker Hub using GitHub repository secrets. -- Builds a production-ready image using the `prod` stage in `Dockerfile`. -- Tags and pushes the final image to Docker Hub with both `latest` and short SHA tags for traceability. - -> [!NOTE] -> For more information about `docker/build-push-action`, refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). - ---- - -### Step 3: Run the workflow - -After you've added your workflow file, it's time to trigger and observe the CI/CD process in action. - -1. Commit and push your workflow file - - - Select "Commit changes…" in the GitHub editor. - - - This push will automatically trigger the GitHub Actions pipeline. - -2. Monitor the workflow execution - - - Go to the Actions tab in your GitHub repository. - - Click into the workflow run to follow each step: **build**, **test**, and (if successful) **push**. - -3. Verify the Docker image on Docker Hub - - - After a successful workflow run, visit your [Docker Hub repositories](https://hub.docker.com/repositories). - - You should see a new image under your repository with: - - Repository name: `${your-repository-name}` - - Tags include: - - `latest` – represents the most recent successful build; ideal for quick testing or deployment. - - `` – a unique identifier based on the commit hash, useful for version tracking, rollbacks, and traceability. - -> [!TIP] Protect your main branch -> To maintain code quality and prevent accidental direct pushes, enable branch protection rules: -> - Navigate to your **GitHub repo → Settings → Branches**. -> - Under Branch protection rules, click **Add rule**. -> - Specify `main` as the branch name. -> - Enable options like: -> - *Require a pull request before merging*. -> - *Require status checks to pass before merging*. -> -> This ensures that only tested and reviewed code is merged into `main` branch. ---- - -## Summary - -In this section, you set up a complete CI/CD pipeline for your containerized Angular application using GitHub Actions. - -Here's what you accomplished: - -- Created a new GitHub repository specifically for your project. -- Generated a secure Docker Hub access token and added it to GitHub as a secret. -- Defined a GitHub Actions workflow that: - - Build your application inside a Docker container. - - Run tests in a consistent, containerized environment. - - Push a production-ready image to Docker Hub if tests pass. -- Triggered and verified the workflow execution through GitHub Actions. -- Confirmed that your image was successfully published to Docker Hub. - -With this setup, your Angular application is now ready for automated testing and deployment across environments — increasing confidence, consistency, and team productivity. - ---- - -## Related resources - -Deepen your understanding of automation and best practices for containerized apps: - -- [Introduction to GitHub Actions](/guides/gha.md) – Learn how GitHub Actions automate your workflows -- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) – Set up container builds with GitHub Actions -- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) – Full reference for writing GitHub workflows -- [Compose file reference](/compose/compose-file/) – Full configuration reference for `compose.yaml` -- [Best practices for writing Dockerfiles](/develop/develop-images/dockerfile_best-practices/) – Optimize your image for performance and security - ---- - -## Next steps - -Next, learn how you can locally test and debug your Angular workloads on Kubernetes before deploying. This helps you ensure your application behaves as expected in a production-like environment, reducing surprises during deployment. diff --git a/content/guides/angular/containerize.md b/content/guides/angular/containerize.md deleted file mode 100644 index 2ba56fc7c92b..000000000000 --- a/content/guides/angular/containerize.md +++ /dev/null @@ -1,529 +0,0 @@ ---- -title: Containerize an Angular Application -linkTitle: Containerize -weight: 10 -keywords: angular, node, image, initialize, build -description: Learn how to containerize an Angular application with Docker by creating an optimized, production-ready image using best practices for performance, security, and scalability. - ---- - -## Prerequisites - -Before you begin, make sure the following tools are installed and available on your system: - -- You have installed the latest version of [Docker Desktop](/get-started/get-docker.md). -- You have a [git client](https://git-scm.com/downloads). The examples in this section use a command-line based git client, but you can use any client. - -> **New to Docker?** -> Start with the [Docker basics](/get-started/docker-concepts/the-basics/what-is-a-container.md) guide to get familiar with key concepts like images, containers, and Dockerfiles. - ---- - -## Overview - -This guide walks you through the complete process of containerizing an Angular application with Docker. You’ll learn how to create a production-ready Docker image using best practices that improve performance, security, scalability, and deployment efficiency. - -By the end of this guide, you will: - -- Containerize an Angular application using Docker. -- Create and optimize a Dockerfile for production builds. -- Use multi-stage builds to minimize image size. -- Serve the application efficiently with a custom Nginx configuration. -- Build secure and maintainable Docker images by following best practices. - ---- - -## Get the sample application - -Clone the sample application to use with this guide. Open a terminal, navigate to the directory where you want to work, and run the following command -to clone the git repository: - -```console -$ git clone https://github.com/kristiyan-velkov/docker-angular-sample -``` ---- - -## Build the Docker image - -Angular is a front-end framework that compiles into static assets, so the Dockerfile uses a multi-stage build: one stage compiles the app with Node.js, and a second minimal stage serves the static output with Nginx. - -> [!TIP] -> -> [Gordon](/ai/gordon/), Docker's AI assistant, can generate Docker assets for your project. Ask Gordon to create a Dockerfile, Compose file, and `.dockerignore` tailored to your application. - -### Step 1: Create the Dockerfile - -Before creating a Dockerfile, you need to choose a base image. You can either use the [Node.js Official Image](https://hub.docker.com/_/node) or a Docker Hardened Image (DHI) from the [Hardened Image catalog](https://hub.docker.com/hardened-images/catalog). - -Choosing DHI offers the advantage of a production-ready image that is lightweight and secure. For more information, see [Docker Hardened Images](https://docs.docker.com/dhi/). - -> [!IMPORTANT] -> This guide uses a stable Node.js LTS image tag that is considered secure when the guide is written. Because new releases and security patches are published regularly, the tag shown here may no longer be the safest option when you follow the guide. Always review the latest available image tags and select a secure, up-to-date version before building or deploying your application. -> -> Official Node.js Docker Images: https://hub.docker.com/_/node - -{{< tabs >}} -{{< tab name="Using Docker Hardened Images" >}} -Docker Hardened Images (DHIs) are available for Node.js in the [Docker Hardened Images catalog](https://hub.docker.com/hardened-images/catalog/dhi/node). Docker Hardened Images are freely available to everyone with no subscription required. You can pull and use them like any other Docker image after signing in to the DHI registry. For more information, see the [DHI quickstart](/dhi/get-started/) guide. - -1. Sign in to the DHI registry: - ```console - $ docker login dhi.io - ``` - -2. Pull the Node.js DHI (check the catalog for available versions): - ```console - $ docker pull dhi.io/node:24-alpine3.22-dev - ``` - -In the following Dockerfile, the `FROM` instruction uses `dhi.io/node:24-alpine3.22-dev` as the base image. - -```dockerfile -# ========================================= -# Stage 1: Build the Angular Application -# ========================================= - -# Use a lightweight DHI Node.js image for building -FROM dhi.io/node:24-alpine3.22-dev AS builder - -# Set the working directory inside the container -WORKDIR /app - -# Copy package-related files first to leverage Docker's caching mechanism -COPY package.json package-lock.json* ./ - -# Install project dependencies using npm ci (ensures a clean, reproducible install) -RUN --mount=type=cache,target=/root/.npm npm ci - -# Copy the rest of the application source code into the container -COPY . . - -# Build the Angular application -RUN npm run build - -# ========================================= -# Stage 2: Prepare Nginx to Serve Static Files -# ========================================= - -FROM dhi.io/nginx:1.28.0-alpine3.21-dev AS runner - -# Copy custom Nginx config -COPY nginx.conf /etc/nginx/nginx.conf - -# Copy the static build output from the build stage to Nginx's default HTML serving directory -COPY --chown=nginx:nginx --from=builder /app/dist/*/browser /usr/share/nginx/html - -# Use a non-root user for security best practices -USER nginx - -# Expose port 8080 to allow HTTP traffic -# Note: The default Nginx container now listens on port 8080 instead of 80 -EXPOSE 8080 - -# Start Nginx directly with custom config -ENTRYPOINT ["nginx", "-c", "/etc/nginx/nginx.conf"] -CMD ["-g", "daemon off;"] -``` - -{{< /tab >}} -{{< tab name="Using the Docker Official Image" >}} - -Create a file named `Dockerfile` with the following contents: - -```dockerfile -# ========================================= -# Stage 1: Build the Angular Application -# ========================================= -ARG NODE_VERSION=24.12.0-alpine -ARG NGINX_VERSION=alpine3.22 - -# Use a lightweight Node.js image for building (customizable via ARG) -FROM node:${NODE_VERSION} AS builder - -# Set the working directory inside the container -WORKDIR /app - -# Copy package-related files first to leverage Docker's caching mechanism -COPY package.json *package-lock.json* ./ - -# Install project dependencies using npm ci (ensures a clean, reproducible install) -RUN --mount=type=cache,target=/root/.npm npm ci - -# Copy the rest of the application source code into the container -COPY . . - -# Build the Angular application -RUN npm run build - -# ========================================= -# Stage 2: Prepare Nginx to Serve Static Files -# ========================================= - -FROM nginxinc/nginx-unprivileged:${NGINX_VERSION} AS runner - -# Copy custom Nginx config -COPY nginx.conf /etc/nginx/nginx.conf - -# Copy the static build output from the build stage to Nginx's default HTML serving directory -COPY --chown=nginx:nginx --from=builder /app/dist/*/browser /usr/share/nginx/html - -# Use a built-in non-root user for security best practices -USER nginx - -# Expose port 8080 to allow HTTP traffic -# Note: The default Nginx container now listens on port 8080 instead of 80 -EXPOSE 8080 - -# Start Nginx directly with custom config -ENTRYPOINT ["nginx", "-c", "/etc/nginx/nginx.conf"] -CMD ["-g", "daemon off;"] -``` - -> [!NOTE] -> We are using nginx-unprivileged instead of the standard Nginx image to follow security best practices. -> Running as a non-root user in the final image: ->- Reduces the attack surface ->- Aligns with Docker’s recommendations for container hardening ->- Helps comply with stricter security policies in production environments - -{{< /tab >}} -{{< /tabs >}} - -### Step 2: Create the compose.yaml file - -Create a file named `compose.yaml` with the following contents: - -```yaml {collapse=true,title=compose.yaml} -services: - server: - build: - context: . - ports: - - 8080:8080 -``` - -### Step 3: Create the .dockerignore file - -The `.dockerignore` file tells Docker which files and folders to exclude when building the image. - -> [!NOTE] ->This helps: ->- Reduce image size ->- Speed up the build process ->- Prevent sensitive or unnecessary files (like `.env`, `.git`, or `node_modules`) from being added to the final image. -> -> To learn more, visit the [.dockerignore reference](/reference/dockerfile.md#dockerignore-file). - -Create a file named `.dockerignore` with the following contents: - -```dockerignore -# ================================ -# Node and build output -# ================================ -node_modules -dist -out-tsc -.angular -.cache -.tmp - -# ================================ -# Testing & Coverage -# ================================ -coverage -jest -cypress -cypress/screenshots -cypress/videos -reports -playwright-report -.vite -.vitepress - -# ================================ -# Environment & log files -# ================================ -*.env* -!*.env.production -*.log -*.tsbuildinfo - -# ================================ -# IDE & OS-specific files -# ================================ -.vscode -.idea -.DS_Store -Thumbs.db -*.swp - -# ================================ -# Version control & CI files -# ================================ -.git -.gitignore - -# ================================ -# Docker & local orchestration -# ================================ -Dockerfile -Dockerfile.* -.dockerignore -docker-compose.yml -docker-compose*.yml - -# ================================ -# Miscellaneous -# ================================ -*.bak -*.old -*.tmp -``` - -### Step 4: Create the `nginx.conf` file - -To serve your Angular application efficiently inside the container, you’ll configure Nginx with a custom setup. This configuration is optimized for performance, browser caching, gzip compression, and support for client-side routing. - -Create a file named `nginx.conf` in the root of your project directory, and add the following content: - -> [!NOTE] -> To learn more about configuring Nginx, see the [official Nginx documentation](https://nginx.org/en/docs/). - - -```nginx -worker_processes auto; - -pid /tmp/nginx.pid; - -events { - worker_connections 1024; -} - -http { - include /etc/nginx/mime.types; - default_type application/octet-stream; - - client_body_temp_path /tmp/client_temp; - proxy_temp_path /tmp/proxy_temp_path; - fastcgi_temp_path /tmp/fastcgi_temp; - uwsgi_temp_path /tmp/uwsgi_temp; - scgi_temp_path /tmp/scgi_temp; - - # Logging - access_log off; - error_log /dev/stderr warn; - - # Performance - sendfile on; - tcp_nopush on; - tcp_nodelay on; - keepalive_timeout 65; - keepalive_requests 1000; - - # Compression - gzip on; - gzip_vary on; - gzip_proxied any; - gzip_min_length 256; - gzip_comp_level 6; - gzip_types - text/plain - text/css - text/xml - text/javascript - application/javascript - application/x-javascript - application/json - application/xml - application/xml+rss - font/ttf - font/otf - image/svg+xml; - - server { - listen 8080; - server_name localhost; - - root /usr/share/nginx/html; - index index.html; - - # Angular Routing - location / { - try_files $uri $uri/ /index.html; - } - - # Static Assets Caching - location ~* \.(?:ico|css|js|gif|jpe?g|png|woff2?|eot|ttf|svg|map)$ { - expires 1y; - access_log off; - add_header Cache-Control "public, immutable"; - } - - # Optional: Explicit asset route - location /assets/ { - expires 1y; - add_header Cache-Control "public, immutable"; - } - } -} -``` - -### Step 5: Build the Angular application image - -With your custom configuration in place, you're now ready to build the Docker image for your Angular application. - -The updated setup includes: - -- The updated setup includes a clean, production-ready Nginx configuration tailored specifically for Angular. -- Efficient multi-stage Docker build, ensuring a small and secure final image. - -After completing the previous steps, your project directory should now contain the following files: - -```text -├── docker-angular-sample/ -│ ├── Dockerfile -│ ├── .dockerignore -│ ├── compose.yaml -│ └── nginx.conf -``` - -Now that your Dockerfile is configured, you can build the Docker image for your Angular application. - -> [!NOTE] -> The `docker build` command packages your application into an image using the instructions in the Dockerfile. It includes all necessary files from the current directory (called the [build context](/build/concepts/context/#what-is-a-build-context)). - -Run the following command from the root of your project: - -```console -$ docker build --tag docker-angular-sample . -``` - -What this command does: -- Uses the Dockerfile in the current directory (.) -- Packages the application and its dependencies into a Docker image -- Tags the image as docker-angular-sample so you can reference it later - - -### Step 6: View local images - -After building your Docker image, you can check which images are available on your local machine using either the Docker CLI or [Docker Desktop](/manuals/desktop/use-desktop/images.md). Since you're already working in the terminal, let's use the Docker CLI. - -To list all locally available Docker images, run the following command: - -```console -$ docker images -``` - -Example Output: - -```shell -REPOSITORY TAG IMAGE ID CREATED SIZE -docker-angular-sample latest 34e66bdb9d40 14 seconds ago 76.4MB -``` - -This output provides key details about your images: - -- **Repository** – The name assigned to the image. -- **Tag** – A version label that helps identify different builds (e.g., latest). -- **Image ID** – A unique identifier for the image. -- **Created** – The timestamp indicating when the image was built. -- **Size** – The total disk space used by the image. - -If the build was successful, you should see `docker-angular-sample` image listed. - ---- - -## Run the containerized application - -In the previous step, you created a Dockerfile for your Angular application and built a Docker image using the docker build command. Now it’s time to run that image in a container and verify that your application works as expected. - - -Inside the `docker-angular-sample` directory, run the following command in a -terminal. - -```console -$ docker compose up --build -``` - -Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see a simple Angular web application. - -Press `ctrl+c` in the terminal to stop your application. - -### Run the application in the background - -You can run the application detached from the terminal by adding the `-d` -option. Inside the `docker-angular-sample` directory, run the following command -in a terminal. - -```console -$ docker compose up --build -d -``` - -Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see your Angular application running in the browser. - - -To confirm that the container is running, use `docker ps` command: - -```console -$ docker ps -``` - -This will list all active containers along with their ports, names, and status. Look for a container exposing port 8080. - -Example Output: - -```shell -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -eb13026806d1 docker-angular-sample-server "nginx -c /etc/nginx…" About a minute ago Up About a minute 0.0.0.0:8080->8080/tcp docker-angular-sample-server-1 -``` - - -To stop the application, run: - -```console -$ docker compose down -``` - - -> [!NOTE] -> For more information about Compose commands, see the [Compose CLI -> reference](/reference/cli/docker/compose/). - ---- - -## Summary - -In this guide, you learned how to containerize, build, and run an Angular application using Docker. By following best practices, you created a secure, optimized, and production-ready setup. - -What you accomplished: -- Created a multi-stage `Dockerfile` that compiles the Angular application and serves the static files using Nginx. -- Created a `.dockerignore` file to exclude unnecessary files and keep the image clean and efficient. -- Built your Docker image using `docker build`. -- Ran the container using `docker compose up`, both in the foreground and in detached mode. -- Verified that the app was running by visiting [http://localhost:8080](http://localhost:8080). -- Learned how to stop the containerized application using `docker compose down`. - -You now have a fully containerized Angular application, running in a Docker container, and ready for deployment across any environment with confidence and consistency. - ---- - -## Related resources - -Explore official references and best practices to sharpen your Docker workflow: - -- [Multi-stage builds](/build/building/multi-stage/) – Learn how to separate build and runtime stages. -- [Best practices for writing Dockerfiles](/develop/develop-images/dockerfile_best-practices/) – Write efficient, maintainable, and secure Dockerfiles. -- [Build context in Docker](/build/concepts/context/) – Learn how context affects image builds. -- [`docker build` CLI reference](/reference/cli/docker/image/build/) – Build Docker images from a Dockerfile. -- [`docker images` CLI reference](/reference/cli/docker/image/ls/) – Manage and inspect local Docker images. -- [`docker compose up` CLI reference](/reference/cli/docker/compose/up/) – Start and run multi-container applications. -- [`docker compose down` CLI reference](/reference/cli/docker/compose/down/) – Stop and remove containers, networks, and volumes. - ---- - -## Next steps - -With your Angular application now containerized, you're ready to move on to the next step. - -In the next section, you'll learn how to develop your application using Docker containers, enabling a consistent, isolated, and reproducible development environment across any machine. - diff --git a/content/guides/angular/deploy.md b/content/guides/angular/deploy.md deleted file mode 100644 index f7b938449f73..000000000000 --- a/content/guides/angular/deploy.md +++ /dev/null @@ -1,201 +0,0 @@ ---- -title: Test your Angular deployment -linkTitle: Test your deployment -weight: 60 -keywords: deploy, kubernetes, angular -description: Learn how to deploy locally to test and debug your Kubernetes deployment - ---- - -## Prerequisites - -Before you begin, make sure you’ve completed the following: -- Complete all the previous sections of this guide, starting with [Containerize Angular application](containerize.md). -- [Enable Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. - -> **New to Kubernetes?** -> Visit the [Kubernetes basics tutorial](https://kubernetes.io/docs/tutorials/kubernetes-basics/) to get familiar with how clusters, pods, deployments, and services work. - ---- - -## Overview - -This section guides you through deploying your containerized Angular application locally using [Docker Desktop’s built-in Kubernetes](/desktop/kubernetes/). Running your app in a local Kubernetes cluster closely simulates a real production environment, enabling you to test, validate, and debug your workloads with confidence before promoting them to staging or production. - ---- - -## Create a Kubernetes YAML file - -Follow these steps to define your deployment configuration: - -1. In the root of your project, create a new file named: angular-sample-kubernetes.yaml - -2. Open the file in your IDE or preferred text editor. - -3. Add the following configuration, and be sure to replace `{DOCKER_USERNAME}` and `{DOCKERHUB_PROJECT_NAME}` with your actual Docker Hub username and repository name from the previous [Automate your builds with GitHub Actions](configure-github-actions.md). - - -```yaml -apiVersion: apps/v1 -kind: Deployment -metadata: - name: angular-sample - namespace: default -spec: - replicas: 1 - selector: - matchLabels: - app: angular-sample - template: - metadata: - labels: - app: angular-sample - spec: - containers: - - name: angular-container - image: {DOCKER_USERNAME}/{DOCKERHUB_PROJECT_NAME}:latest - imagePullPolicy: Always - ports: - - containerPort: 8080 - resources: - limits: - cpu: "500m" - memory: "256Mi" - requests: - cpu: "250m" - memory: "128Mi" ---- -apiVersion: v1 -kind: Service -metadata: - name: angular-sample-service - namespace: default -spec: - type: NodePort - selector: - app: angular-sample - ports: - - port: 8080 - targetPort: 8080 - nodePort: 30001 -``` - -This manifest defines two key Kubernetes resources, separated by `---`: - -- Deployment - Deploys a single replica of your Angular application inside a pod. The pod uses the Docker image built and pushed by your GitHub Actions CI/CD workflow - (refer to [Automate your builds with GitHub Actions](configure-github-actions.md)). - The container listens on port `8080`, which is typically used by [Nginx](https://nginx.org/en/docs/) to serve your production Angular app. - -- Service (NodePort) - Exposes the deployed pod to your local machine. - It forwards traffic from port `30001` on your host to port `8080` inside the container. - This lets you access the application in your browser at [http://localhost:30001](http://localhost:30001). - -> [!NOTE] -> To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). - ---- - -## Deploy and check your application - -Follow these steps to deploy your containerized Angular app into a local Kubernetes cluster and verify that it’s running correctly. - -### Step 1. Apply the Kubernetes configuration - -In your terminal, navigate to the directory where your `angular-sample-kubernetes.yaml` file is located, then deploy the resources using: - -```console - $ kubectl apply -f angular-sample-kubernetes.yaml -``` - -If everything is configured properly, you’ll see confirmation that both the Deployment and the Service were created: - -```shell - deployment.apps/angular-sample created - service/angular-sample-service created -``` - -This confirms that both the Deployment and the Service were successfully created and are now running inside your local cluster. - -### Step 2. Check the deployment status - -Run the following command to check the status of your deployment: - -```console - $ kubectl get deployments -``` - -You should see output similar to the following: - -```shell - NAME READY UP-TO-DATE AVAILABLE AGE - angular-sample 1/1 1 1 14s -``` - -This confirms that your pod is up and running with one replica available. - -### Step 3. Verify the service exposure - -Check if the NodePort service is exposing your app to your local machine: - -```console -$ kubectl get services -``` - -You should see something like: - -```shell -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -angular-sample-service NodePort 10.100.185.105 8080:30001/TCP 1m -``` - -This output confirms that your app is available via NodePort on port 30001. - -### Step 4. Access your app in the browser - -Open your browser and navigate to [http://localhost:30001](http://localhost:30001). - -You should see your production-ready Angular Sample application running — served by your local Kubernetes cluster. - -### Step 5. Clean up Kubernetes resources - -Once you're done testing, you can delete the deployment and service using: - -```console - $ kubectl delete -f angular-sample-kubernetes.yaml -``` - -Expected output: - -```shell - deployment.apps "angular-sample" deleted - service "angular-sample-service" deleted -``` - -This ensures your cluster stays clean and ready for the next deployment. - ---- - -## Summary - -In this section, you learned how to deploy your Angular application to a local Kubernetes cluster using Docker Desktop. This setup allows you to test and debug your containerized app in a production-like environment before deploying it to the cloud. - -What you accomplished: - -- Created a Kubernetes Deployment and NodePort Service for your Angular app -- Used `kubectl apply` to deploy the application locally -- Verified the app was running and accessible at `http://localhost:30001` -- Cleaned up your Kubernetes resources after testing - ---- - -## Related resources - -Explore official references and best practices to sharpen your Kubernetes deployment workflow: - -- [Kubernetes documentation](https://kubernetes.io/docs/home/) – Learn about core concepts, workloads, services, and more. -- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) – Use Docker Desktop’s built-in Kubernetes support for local testing and development. -- [`kubectl` CLI reference](https://kubernetes.io/docs/reference/kubectl/) – Manage Kubernetes clusters from the command line. -- [Kubernetes Deployment resource](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) – Understand how to manage and scale applications using Deployments. -- [Kubernetes Service resource](https://kubernetes.io/docs/concepts/services-networking/service/) – Learn how to expose your application to internal and external traffic. \ No newline at end of file diff --git a/content/guides/angular/develop.md b/content/guides/angular/develop.md deleted file mode 100644 index 02c65ac6ba92..000000000000 --- a/content/guides/angular/develop.md +++ /dev/null @@ -1,178 +0,0 @@ ---- -title: Use containers for Angular development -linkTitle: Develop your app -weight: 30 -keywords: angular, development, node -description: Learn how to develop your Angular application locally using containers. - ---- - -## Prerequisites - -Complete [Containerize Angular application](containerize.md). - ---- - -## Overview - -In this section, you'll learn how to set up both production and development environments for your containerized Angular application using Docker Compose. This setup allows you to serve a static production build via Nginx and to develop efficiently inside containers using a live-reloading dev server with Compose Watch. - -You’ll learn how to: -- Configure separate containers for production and development -- Enable automatic file syncing using Compose Watch in development -- Debug and live-preview your changes in real-time without manual rebuilds - ---- - -## Automatically update services (development mode) - -Use Compose Watch to automatically sync source file changes into your containerized development environment. This provides a seamless, efficient development experience without restarting or rebuilding containers manually. - -## Step 1: Create a development Dockerfile - -Create a file named `Dockerfile.dev` in your project root with the following content: - -```dockerfile -# ========================================= -# Stage 1: Development - Angular Application -# ========================================= - -# Define the Node.js version to use (Alpine for a small footprint) -ARG NODE_VERSION=24.12.0-alpine - -# Set the base image for development -FROM node:${NODE_VERSION} AS dev - -# Set environment variable to indicate development mode -ENV NODE_ENV=development - -# Set the working directory inside the container -WORKDIR /app - -# Copy only the dependency files first to optimize Docker caching -COPY package.json package-lock.json* ./ - -# Install dependencies using npm with caching to speed up subsequent builds -RUN --mount=type=cache,target=/root/.npm npm install - -# Copy all application source files into the container -COPY . . - -# Expose the port Angular uses for the dev server (default is 4200) -EXPOSE 4200 - -# Start the Angular dev server and bind it to all network interfaces -CMD ["npm", "start", "--", "--host=0.0.0.0"] - -``` - -This file sets up a lightweight development environment for your Angular application using the dev server. - - -### Step 2: Update your `compose.yaml` file - -Open your `compose.yaml` file and define two services: one for production (`angular-prod`) and one for development (`angular-dev`). - -Here’s an example configuration for an Angular application: - -```yaml -services: - angular-prod: - build: - context: . - dockerfile: Dockerfile - image: docker-angular-sample - ports: - - "8080:8080" - - angular-dev: - build: - context: . - dockerfile: Dockerfile.dev - ports: - - "4200:4200" - develop: - watch: - - action: sync - path: . - target: /app -``` -- The `angular-prod` service builds and serves your static production app using Nginx. -- The `angular-dev` service runs your Angular development server with live reload and hot module replacement. -- `watch` triggers file sync with Compose Watch. - -> [!NOTE] -> For more details, see the official guide: [Use Compose Watch](/manuals/compose/how-tos/file-watch.md). - -After completing the previous steps, your project directory should now contain the following files: - -```text -├── docker-angular-sample/ -│ ├── Dockerfile -│ ├── Dockerfile.dev -│ ├── .dockerignore -│ ├── compose.yaml -│ └── nginx.conf -``` - -### Step 4: Start Compose Watch - -Run the following command from the project root to start the container in watch mode - -```console -$ docker compose watch angular-dev -``` - -### Step 5: Test Compose Watch with Angular - -To verify that Compose Watch is working correctly: - -1. Open the `src/app/app.component.html` file in your text editor. - -2. Locate the following line: - - ```html -

Docker Angular Sample Application

- ``` - -3. Change it to: - - ```html -

Hello from Docker Compose Watch

- ``` - -4. Save the file. - -5. Open your browser at [http://localhost:4200](http://localhost:4200). - -You should see the updated text appear instantly, without needing to rebuild the container manually. This confirms that file watching and automatic synchronization are working as expected. - ---- - -## Summary - -In this section, you set up a complete development and production workflow for your Angular application using Docker and Docker Compose. - -Here’s what you accomplished: -- Created a `Dockerfile.dev` to streamline local development with hot reloading -- Defined separate `angular-dev` and `angular-prod` services in your `compose.yaml` file -- Enabled real-time file syncing using Compose Watch for a smoother development experience -- Verified that live updates work seamlessly by modifying and previewing a component - -With this setup, you're now equipped to build, run, and iterate on your Angular app entirely within containers—efficiently and consistently across environments. - ---- - -## Related resources - -Deepen your knowledge and improve your containerized development workflow with these guides: - -- [Using Compose Watch](/manuals/compose/how-tos/file-watch.md) – Automatically sync source changes during development -- [Multi-stage builds](/manuals/build/building/multi-stage.md) – Create efficient, production-ready Docker images -- [Dockerfile best practices](/build/building/best-practices/) – Write clean, secure, and optimized Dockerfiles. -- [Compose file reference](/compose/compose-file/) – Learn the full syntax and options available for configuring services in `compose.yaml`. -- [Docker volumes](/storage/volumes/) – Persist and manage data between container runs - -## Next steps - -In the next section, you'll learn how to run unit tests for your Angular application inside Docker containers. This ensures consistent testing across all environments and removes dependencies on local machine setup. diff --git a/content/guides/angular/run-tests.md b/content/guides/angular/run-tests.md deleted file mode 100644 index 309097111465..000000000000 --- a/content/guides/angular/run-tests.md +++ /dev/null @@ -1,137 +0,0 @@ ---- -title: Run Angular tests in a container -linkTitle: Run your tests -weight: 40 -keywords: angular, test, jasmine -description: Learn how to run your Angular tests in a container. - ---- - -## Prerequisites - -Complete all the previous sections of this guide, starting with [Containerize Angular application](containerize.md). - -## Overview - -Testing is a critical part of the development process. In this section, you'll learn how to: - -- Run Jasmine unit tests using the Angular CLI inside a Docker container. -- Use Docker Compose to isolate your test environment. -- Ensure consistency between local and container-based testing. - - -The `docker-angular-sample` project comes pre-configured with Jasmine, so you can get started quickly without extra setup. - ---- - -## Run tests during development - -The `docker-angular-sample` application includes a sample test file at the following location: - -```console -$ src/app/app.component.spec.ts -``` - -This test uses Jasmine to validate the AppComponent logic. - -### Step 1: Update compose.yaml - -Add a new service named `angular-test` to your `compose.yaml` file. This service allows you to run your test suite in an isolated, containerized environment. - -```yaml {hl_lines="22-26",linenos=true} -services: - angular-dev: - build: - context: . - dockerfile: Dockerfile.dev - ports: - - "5173:5173" - develop: - watch: - - action: sync - path: . - target: /app - - angular-prod: - build: - context: . - dockerfile: Dockerfile - image: docker-angular-sample - ports: - - "8080:8080" - - angular-test: - build: - context: . - dockerfile: Dockerfile.dev - command: ["npm", "run", "test"] - -``` - -The angular-test service reuses the same `Dockerfile.dev` used for [development](develop.md) and overrides the default command to run tests with `npm run test`. This setup ensures a consistent test environment that matches your local development configuration. - - -After completing the previous steps, your project directory should contain the following files: - -```text -├── docker-angular-sample/ -│ ├── Dockerfile -│ ├── Dockerfile.dev -│ ├── .dockerignore -│ ├── compose.yaml -│ └── nginx.conf -``` - -### Step 2: Run the tests - -To execute your test suite inside the container, run the following command from your project root: - -```console -$ docker compose run --rm angular-test -``` - -This command will: -- Start the `angular-test` service defined in your `compose.yaml` file. -- Execute the `npm run test` script using the same environment as development. -- Automatically removes the container after tests complete, using the [`docker compose run --rm`](/reference/cli/docker/compose/run/) command. - -You should see output similar to the following: - -```shell -Test Suites: 1 passed, 1 total -Tests: 3 passed, 3 total -Snapshots: 0 total -Time: 1.529 s -``` - -> [!NOTE] -> For more information about Compose commands, see the [Compose CLI -> reference](/reference/cli/docker/compose/). - ---- - -## Summary - -In this section, you learned how to run unit tests for your Angular application inside a Docker container using Jasmine and Docker Compose. - -What you accomplished: -- Created a `angular-test` service in `compose.yaml` to isolate test execution. -- Reused the development `Dockerfile.dev` to ensure consistency between dev and test environments. -- Ran tests inside the container using `docker compose run --rm angular-test`. -- Ensured reliable, repeatable testing across environments without depending on your local machine setup. - ---- - -## Related resources - -Explore official references and best practices to sharpen your Docker testing workflow: - -- [Dockerfile reference](/reference/dockerfile/) – Understand all Dockerfile instructions and syntax. -- [Best practices for writing Dockerfiles](/develop/develop-images/dockerfile_best-practices/) – Write efficient, maintainable, and secure Dockerfiles. -- [Compose file reference](/compose/compose-file/) – Learn the full syntax and options available for configuring services in `compose.yaml`. -- [`docker compose run` CLI reference](/reference/cli/docker/compose/run/) – Run one-off commands in a service container. ---- - -## Next steps - -Next, you’ll learn how to set up a CI/CD pipeline using GitHub Actions to automatically build and test your Angular application in a containerized environment. This ensures your code is validated on every push or pull request, maintaining consistency and reliability across your development workflow. diff --git a/content/guides/azure-pipelines.md b/content/guides/azure-pipelines.md index 358133c1d8c7..098cfd319f01 100644 --- a/content/guides/azure-pipelines.md +++ b/content/guides/azure-pipelines.md @@ -5,7 +5,7 @@ summary: | Learn how to automate Docker image build and push using Azure Pipelines. keywords: azure pipelines, azure devops, ci/cd, docker hub, build and push, automation params: - tags: [devops] + tags: [cicd] time: 10 minutes --- @@ -308,5 +308,5 @@ With this Azure Pipelines CI setup, you get: - [Azure Pipelines Documentation](https://learn.microsoft.com/en-us/azure/devops/pipelines/?view=azure-devops): Comprehensive guide to configuring and managing CI/CD pipelines in Azure DevOps. - [Docker Task for Azure Pipelines](https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/build/docker): Detailed reference for using the Docker task in Azure Pipelines to build and push images. -- [Docker Buildx Bake](/manuals/build/bake/_index.md): Explore Docker's advanced build tool for complex, multi-stage, and multi-platform build setups. See also the [Mastering Buildx Bake Guide](/guides/bake/index.md) for practical examples and best practices. +- [Docker Buildx Bake](/manuals/build/bake/_index.md): Explore Docker's advanced build tool for complex, multi-stage, and multi-platform build setups. See also the [Mastering Buildx Bake Guide](/guides/bake/) for practical examples and best practices. - [Docker Build Cloud](/guides/docker-build-cloud/_index.md): Learn about Docker's managed build service for faster, scalable, and multi-platform image builds in the cloud. diff --git a/content/guides/bake/index.md b/content/guides/bake.md similarity index 97% rename from content/guides/bake/index.md rename to content/guides/bake.md index bf03cbe7b413..9f70755bfa61 100644 --- a/content/guides/bake/index.md +++ b/content/guides/bake.md @@ -6,9 +6,8 @@ description: > summary: > Learn to automate Docker builds and testing with declarative configurations using Buildx Bake. keywords: bake, buildx, multi-platform, build configuration, hcl, automation -tags: [devops] -languages: [go] params: + tags: [cicd] time: 30 minutes image: /images/guides/bake.webp --- @@ -441,9 +440,9 @@ build/ └── bin ├── bakeme ├── linux_amd64 - │   └── bakeme + │ └── bakeme ├── linux_arm64 - │   └── bakeme + │ └── bakeme └── linux_riscv64 └── bakeme @@ -476,17 +475,17 @@ $ tree build/ build/ └── bin ├── debug - │   ├── linux_amd64 - │   │   └── bakeme - │   ├── linux_arm64 - │   │   └── bakeme - │   └── linux_riscv64 - │   └── bakeme + │ ├── linux_amd64 + │ │ └── bakeme + │ ├── linux_arm64 + │ │ └── bakeme + │ └── linux_riscv64 + │ └── bakeme └── release ├── linux_amd64 - │   └── bakeme + │ └── bakeme ├── linux_arm64 - │   └── bakeme + │ └── bakeme └── linux_riscv64 └── bakeme diff --git a/content/guides/bun/_index.md b/content/guides/bun/_index.md index b50e184c32e1..537c91740a62 100644 --- a/content/guides/bun/_index.md +++ b/content/guides/bun/_index.md @@ -5,12 +5,17 @@ title: Bun language-specific guide summary: | Learn how to containerize JavaScript applications with the Bun runtime. linkTitle: Bun -languages: [js] -tags: [dhi] +aliases: + - /guides/bun/configure-ci-cd/ + - /guides/bun/containerize/ + - /guides/bun/deploy/ + - /guides/bun/develop/ params: + tags: [cicd] time: 10 minutes --- + The Bun getting started guide teaches you how to create a containerized Bun application using Docker. > **Acknowledgment** @@ -32,3 +37,498 @@ The Bun getting started guide teaches you how to create a containerized Bun appl After completing the Bun getting started modules, you should be able to containerize your own Bun application based on the examples and instructions provided in this guide. Start by containerizing an existing Bun application. + +## Containerize a Bun application + +### Prerequisites + +* You have a [Git client](https://git-scm.com/downloads). The examples in this section use a command-line based Git client, but you can use any client. + +### Overview + +For a long time, Node.js has been the de-facto runtime for server-side +JavaScript applications. Recent years have seen a rise in new alternative +runtimes in the ecosystem, including [Bun website](https://bun.sh/). Like +Node.js, Bun is a JavaScript runtime. Bun is a comparatively lightweight +runtime that is designed to be fast and efficient. + +Why develop Bun applications with Docker? Having multiple runtimes to choose +from is great. But as the number of runtimes increases, it becomes challenging +to manage the different runtimes and their dependencies consistently across +environments. This is where Docker comes in. Creating and destroying containers +on demand is a great way to manage the different runtimes and their +dependencies. Also, as it's fairly a new runtime, getting a consistent +development environment for Bun can be challenging. Docker can help you set up +a consistent development environment for Bun. + +### Get the sample application + +Clone the sample application to use with this guide. Open a terminal, change +directory to a directory that you want to work in, and run the following +command to clone the repository: + +```console +$ git clone https://github.com/dockersamples/bun-docker.git && cd bun-docker +``` + +You should now have the following contents in your `bun-docker` directory. + +```text +├── bun-docker/ +│ ├── compose.yml +│ ├── Dockerfile +│ ├── LICENSE +│ ├── server.js +│ └── README.md +``` + +### Create a Dockerfile + +Before creating a Dockerfile, you need to choose a base image. You can either use the [Bun Docker Official Image](https://hub.docker.com/r/oven/bun) or a Docker Hardened Image (DHI) from the [Hardened Image catalog](https://hub.docker.com/hardened-images/catalog). + +Choosing DHI offers the advantage of a production-ready image that is lightweight and secure. For more information, see [Docker Hardened Images](https://docs.docker.com/dhi/). + +{{< tabs >}} +{{< tab name="Using Docker Hardened Images" >}} + +Docker Hardened Images (DHIs) are available for Bun in the [Docker Hardened Images catalog](https://hub.docker.com/hardened-images/catalog/dhi/bun). You can pull DHIs directly from the `dhi.io` registry. + +1. Sign in to the DHI registry: + ```console + $ docker login dhi.io + ``` + +2. Pull the Bun DHI as `dhi.io/bun:1`. The tag (`1`) in this example refers to the version to the latest 1.x version of Bun. + + ```console + $ docker pull dhi.io/bun:1 + ``` + +For other available versions, refer to the [catalog](https://hub.docker.com/hardened-images/catalog/dhi/bun). + +```dockerfile +# Use the DHI Bun image as the base image +FROM dhi.io/bun:1 + +# Set the working directory in the container +WORKDIR /app + +# Copy the current directory contents into the container at /app +COPY . . + +# Expose the port on which the API will listen +EXPOSE 3000 + +# Run the server when the container launches +CMD ["bun", "server.js"] +``` + +{{< /tab >}} +{{< tab name="Using the official image" >}} + +Using the Docker Official Image is straightforward. In the following Dockerfile, you'll notice that the `FROM` instruction uses `oven/bun` as the base image. + +You can find the image on [Docker Hub](https://hub.docker.com/r/oven/bun). This is the Docker Official Image for Bun created by Oven, the company behind Bun, and it's available on Docker Hub. + +```dockerfile +# Use the official Bun image +FROM oven/bun:latest + +# Set the working directory in the container +WORKDIR /app + +# Copy the current directory contents into the container at /app +COPY . . + +# Expose the port on which the API will listen +EXPOSE 3000 + +# Run the server when the container launches +CMD ["bun", "server.js"] +``` + +{{< /tab >}} +{{< /tabs >}} + +In addition to specifying the base image, the Dockerfile also: + +- Sets the working directory in the container to `/app`. +- Copies the content of the current directory to the `/app` directory in the container. +- Exposes port 3000, where the API is listening for requests. +- And finally, starts the server when the container launches with the command `bun server.js`. + +### Run the application + +Inside the `bun-docker` directory, run the following command in a terminal. + +```console +$ docker compose up --build +``` + +Open a browser and view the application at [http://localhost:3000](http://localhost:3000). You will see a message `{"Status" : "OK"}` in the browser. + +In the terminal, press `ctrl`+`c` to stop the application. + +#### Run the application in the background + +You can run the application detached from the terminal by adding the `-d` +option. Inside the `bun-docker` directory, run the following command +in a terminal. + +```console +$ docker compose up --build -d +``` + +Open a browser and view the application at [http://localhost:3000](http://localhost:3000). + + +In the terminal, run the following command to stop the application. + +```console +$ docker compose down +``` + +### Summary + +In this section, you learned how you can containerize and run your Bun +application using Docker. + +Related information: + + - [Dockerfile reference](/reference/dockerfile.md) + - [.dockerignore file](/reference/dockerfile.md#dockerignore-file) + - [Docker Compose overview](/manuals/compose/_index.md) + - [Compose file reference](/reference/compose-file/_index.md) + - [Docker Hardened Images](/dhi/) + +### Next steps + +In the next section, you'll learn how you can develop your application using +containers. + +## Use containers for Bun development + +### Prerequisites + +Complete [Containerize a Bun application](containerize.md). + +### Overview + +In this section, you'll learn how to set up a development environment for your containerized application. This includes: + +- Configuring Compose to automatically update your running Compose services as you edit and save your code + +### Get the sample application + +Clone the sample application to use with this guide. Open a terminal, change directory to a directory that you want to work in, and run the following command to clone the repository: + +```console +$ git clone https://github.com/dockersamples/bun-docker.git && cd bun-docker +``` + +### Automatically update services + +Use Compose Watch to automatically update your running Compose services as you +edit and save your code. For more details about Compose Watch, see [Use Compose +Watch](/manuals/compose/how-tos/file-watch.md). + +Open your `compose.yml` file in an IDE or text editor and then add the Compose Watch instructions. The following example shows how to add Compose Watch to your `compose.yml` file. + +```yaml {hl_lines="9-12",linenos=true} +services: + server: + image: bun-server + build: + context: . + dockerfile: Dockerfile + ports: + - "3000:3000" + develop: + watch: + - action: rebuild + path: . +``` + +Run the following command to run your application with Compose Watch. + +```console +$ docker compose watch +``` + +Now, if you modify your `server.js` you will see the changes in real time without re-building the image. + +To test it out, open the `server.js` file in your favorite text editor and change the message from `{"Status" : "OK"}` to `{"Status" : "Updated"}`. Save the file and refresh your browser at `http://localhost:3000`. You should see the updated message. + +Press `ctrl+c` in the terminal to stop your application. + +### Summary + +In this section, you also learned how to use Compose Watch to automatically rebuild and run your container when you update your code. + +Related information: + - [Compose file reference](/reference/compose-file/) + - [Compose file watch](/manuals/compose/how-tos/file-watch.md) + - [Multi-stage builds](/manuals/build/building/multi-stage.md) + +### Next steps + +In the next section, you'll take a look at how to set up a CI/CD pipeline using GitHub Actions. + +## Configure CI/CD for your Bun application + +### Prerequisites + +Complete all the previous sections of this guide, starting with [Containerize a Bun application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. + +### Overview + +In this section, you'll learn how to set up and use GitHub Actions to build and test your Docker image as well as push it to Docker Hub. You will complete the following steps: + +1. Create a new repository on GitHub. +2. Define the GitHub Actions workflow. +3. Run the workflow. + +### Step one: Create the repository + +Create a GitHub repository, configure the Docker Hub credentials, and push your source code. + +1. [Create a new repository](https://github.com/new) on GitHub. + +2. Open the repository **Settings**, and go to **Secrets and variables** > + **Actions**. + +3. Create a new **Repository variable** named `DOCKER_USERNAME` and your Docker ID as a value. + +4. Create a new [Personal Access Token (PAT)](/manuals/security/access-tokens.md#create-an-access-token)for Docker Hub. You can name this token `docker-tutorial`. Make sure access permissions include Read and Write. + +5. Add the PAT as a **Repository secret** in your GitHub repository, with the name + `DOCKERHUB_TOKEN`. + +6. In your local repository on your machine, run the following command to change + the origin to the repository you just created. Make sure you change + `your-username` to your GitHub username and `your-repository` to the name of + the repository you created. + + ```console + $ git remote set-url origin https://github.com/your-username/your-repository.git + ``` + +7. Run the following commands to stage, commit, and push your local repository to GitHub. + + ```console + $ git add -A + $ git commit -m "my commit" + $ git push -u origin main + ``` + +### Step two: Set up the workflow + +Set up your GitHub Actions workflow for building, testing, and pushing the image +to Docker Hub. + +1. Go to your repository on GitHub and then select the **Actions** tab. + +2. Select **set up a workflow yourself**. + + This takes you to a page for creating a new GitHub actions workflow file in + your repository, under `.github/workflows/main.yml` by default. + +3. In the editor window, copy and paste the following YAML configuration and commit the changes. + + ```yaml + name: ci + + on: + push: + branches: + - main + + jobs: + build: + runs-on: ubuntu-latest + steps: + - name: Login to Docker Hub + uses: docker/login-action@{{% param "login_action_version" %}} + with: + username: ${{ vars.DOCKER_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} + + - name: Build and push + uses: docker/build-push-action@{{% param "build_push_action_version" %}} + with: + platforms: linux/amd64,linux/arm64 + push: true + tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest + ``` + + For more information about the YAML syntax for `docker/build-push-action`, + refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). + +### Step three: Run the workflow + +Save the workflow file and run the job. + +1. Select **Commit changes...** and push the changes to the `main` branch. + + After pushing the commit, the workflow starts automatically. + +2. Go to the **Actions** tab. It displays the workflow. + + Selecting the workflow shows you the breakdown of all the steps. + +3. When the workflow is complete, go to your + [repositories on Docker Hub](https://hub.docker.com/repositories). + + If you see the new repository in that list, it means the GitHub Actions + successfully pushed the image to Docker Hub. + +### Summary + +In this section, you learned how to set up a GitHub Actions workflow for your Bun application. + +Related information: + +- [Introduction to GitHub Actions](/guides/gha.md) +- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) +- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) + +### Next steps + +Next, learn how you can locally test and debug your workloads on Kubernetes before deploying. + +## Test your Bun deployment + +### Prerequisites + +- Complete all the previous sections of this guide, starting with [Containerize a Bun application](containerize.md). +- [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. + +### Overview + +In this section, you'll learn how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. This allows you to test and debug your workloads on Kubernetes locally before deploying. + +### Create a Kubernetes YAML file + +In your `bun-docker` directory, create a file named +`docker-kubernetes.yml`. Open the file in an IDE or text editor and add +the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker +username and the name of the repository that you created in [Configure CI/CD for +your Bun application](configure-ci-cd.md). + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: docker-bun-demo + namespace: default +spec: + replicas: 1 + selector: + matchLabels: + app: bun-api + template: + metadata: + labels: + app: bun-api + spec: + containers: + - name: bun-api + image: DOCKER_USERNAME/REPO_NAME + imagePullPolicy: Always +--- +apiVersion: v1 +kind: Service +metadata: + name: service-entrypoint + namespace: default +spec: + type: NodePort + selector: + app: bun-api + ports: + - port: 3000 + targetPort: 3000 + nodePort: 30001 +``` + +In this Kubernetes YAML file, there are two objects, separated by the `---`: + + - A Deployment, describing a scalable group of identical pods. In this case, + you'll get just one replica, or copy of your pod. That pod, which is + described under `template`, has just one container in it. The + container is created from the image built by GitHub Actions in [Configure CI/CD for + your Bun application](configure-ci-cd.md). + - A NodePort service, which will route traffic from port 30001 on your host to + port 3000 inside the pods it routes to, allowing you to reach your app + from the network. + +To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). + +### Deploy and check your application + +1. In a terminal, navigate to `bun-docker` and deploy your application to + Kubernetes. + + ```console + $ kubectl apply -f docker-kubernetes.yml + ``` + + You should see output that looks like the following, indicating your Kubernetes objects were created successfully. + + ```text + deployment.apps/docker-bun-demo created + service/service-entrypoint created + ``` + +2. Make sure everything worked by listing your deployments. + + ```console + $ kubectl get deployments + ``` + + Your deployment should be listed as follows: + + ```shell + NAME READY UP-TO-DATE AVAILABLE AGE + docker-bun-demo 1/1 1 1 10s + ``` + + This indicates all one of the pods you asked for in your YAML are up and running. Do the same check for your services. + + ```console + $ kubectl get services + ``` + + You should get output like the following. + + ```shell + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + kubernetes ClusterIP 10.96.0.1 443/TCP 88m + service-entrypoint NodePort 10.105.145.223 3000:30001/TCP 83s + ``` + + In addition to the default `kubernetes` service, you can see your `service-entrypoint` service, accepting traffic on port 30001/TCP. + +3. In a browser, visit the following address. You should see the message `{"Status" : "OK"}`. + + ```console + http://localhost:30001/ + ``` + +4. Run the following command to tear down your application. + + ```console + $ kubectl delete -f docker-kubernetes.yml + ``` + +### Summary + +In this section, you learned how to use Docker Desktop to deploy your Bun application to a fully-featured Kubernetes environment on your development machine. + +Related information: + - [Kubernetes documentation](https://kubernetes.io/docs/home/) + - [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) + - [Swarm mode overview](/manuals/engine/swarm/_index.md) diff --git a/content/guides/bun/configure-ci-cd.md b/content/guides/bun/configure-ci-cd.md deleted file mode 100644 index d1c5b9ddc45e..000000000000 --- a/content/guides/bun/configure-ci-cd.md +++ /dev/null @@ -1,132 +0,0 @@ ---- -title: Configure CI/CD for your Bun application -linkTitle: Configure CI/CD -weight: 40 -keywords: ci/cd, github actions, bun, shiny -description: Learn how to configure CI/CD using GitHub Actions for your Bun application. -aliases: -- /language/bun/configure-ci-cd/ ---- - -## Prerequisites - -Complete all the previous sections of this guide, starting with [Containerize a Bun application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. - -## Overview - -In this section, you'll learn how to set up and use GitHub Actions to build and test your Docker image as well as push it to Docker Hub. You will complete the following steps: - -1. Create a new repository on GitHub. -2. Define the GitHub Actions workflow. -3. Run the workflow. - -## Step one: Create the repository - -Create a GitHub repository, configure the Docker Hub credentials, and push your source code. - -1. [Create a new repository](https://github.com/new) on GitHub. - -2. Open the repository **Settings**, and go to **Secrets and variables** > - **Actions**. - -3. Create a new **Repository variable** named `DOCKER_USERNAME` and your Docker ID as a value. - -4. Create a new [Personal Access Token (PAT)](/manuals/security/access-tokens.md#create-an-access-token)for Docker Hub. You can name this token `docker-tutorial`. Make sure access permissions include Read and Write. - -5. Add the PAT as a **Repository secret** in your GitHub repository, with the name - `DOCKERHUB_TOKEN`. - -6. In your local repository on your machine, run the following command to change - the origin to the repository you just created. Make sure you change - `your-username` to your GitHub username and `your-repository` to the name of - the repository you created. - - ```console - $ git remote set-url origin https://github.com/your-username/your-repository.git - ``` - -7. Run the following commands to stage, commit, and push your local repository to GitHub. - - ```console - $ git add -A - $ git commit -m "my commit" - $ git push -u origin main - ``` - -## Step two: Set up the workflow - -Set up your GitHub Actions workflow for building, testing, and pushing the image -to Docker Hub. - -1. Go to your repository on GitHub and then select the **Actions** tab. - -2. Select **set up a workflow yourself**. - - This takes you to a page for creating a new GitHub actions workflow file in - your repository, under `.github/workflows/main.yml` by default. - -3. In the editor window, copy and paste the following YAML configuration and commit the changes. - - ```yaml - name: ci - - on: - push: - branches: - - main - - jobs: - build: - runs-on: ubuntu-latest - steps: - - name: Login to Docker Hub - uses: docker/login-action@{{% param "login_action_version" %}} - with: - username: ${{ vars.DOCKER_USERNAME }} - password: ${{ secrets.DOCKERHUB_TOKEN }} - - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} - - - name: Build and push - uses: docker/build-push-action@{{% param "build_push_action_version" %}} - with: - platforms: linux/amd64,linux/arm64 - push: true - tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest - ``` - - For more information about the YAML syntax for `docker/build-push-action`, - refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). - -## Step three: Run the workflow - -Save the workflow file and run the job. - -1. Select **Commit changes...** and push the changes to the `main` branch. - - After pushing the commit, the workflow starts automatically. - -2. Go to the **Actions** tab. It displays the workflow. - - Selecting the workflow shows you the breakdown of all the steps. - -3. When the workflow is complete, go to your - [repositories on Docker Hub](https://hub.docker.com/repositories). - - If you see the new repository in that list, it means the GitHub Actions - successfully pushed the image to Docker Hub. - -## Summary - -In this section, you learned how to set up a GitHub Actions workflow for your Bun application. - -Related information: - -- [Introduction to GitHub Actions](/guides/gha.md) -- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) -- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) - -## Next steps - -Next, learn how you can locally test and debug your workloads on Kubernetes before deploying. diff --git a/content/guides/bun/containerize.md b/content/guides/bun/containerize.md deleted file mode 100644 index 08f516c88396..000000000000 --- a/content/guides/bun/containerize.md +++ /dev/null @@ -1,175 +0,0 @@ ---- -title: Containerize a Bun application -linkTitle: Containerize your app -weight: 10 -keywords: bun, containerize, initialize -description: Learn how to containerize a Bun application. -aliases: - - /language/bun/containerize/ ---- - -## Prerequisites - -* You have a [Git client](https://git-scm.com/downloads). The examples in this section use a command-line based Git client, but you can use any client. - -## Overview - -For a long time, Node.js has been the de-facto runtime for server-side -JavaScript applications. Recent years have seen a rise in new alternative -runtimes in the ecosystem, including [Bun website](https://bun.sh/). Like -Node.js, Bun is a JavaScript runtime. Bun is a comparatively lightweight -runtime that is designed to be fast and efficient. - -Why develop Bun applications with Docker? Having multiple runtimes to choose -from is great. But as the number of runtimes increases, it becomes challenging -to manage the different runtimes and their dependencies consistently across -environments. This is where Docker comes in. Creating and destroying containers -on demand is a great way to manage the different runtimes and their -dependencies. Also, as it's fairly a new runtime, getting a consistent -development environment for Bun can be challenging. Docker can help you set up -a consistent development environment for Bun. - -## Get the sample application - -Clone the sample application to use with this guide. Open a terminal, change -directory to a directory that you want to work in, and run the following -command to clone the repository: - -```console -$ git clone https://github.com/dockersamples/bun-docker.git && cd bun-docker -``` - -You should now have the following contents in your `bun-docker` directory. - -```text -├── bun-docker/ -│ ├── compose.yml -│ ├── Dockerfile -│ ├── LICENSE -│ ├── server.js -│ └── README.md -``` - -## Create a Dockerfile - -Before creating a Dockerfile, you need to choose a base image. You can either use the [Bun Docker Official Image](https://hub.docker.com/r/oven/bun) or a Docker Hardened Image (DHI) from the [Hardened Image catalog](https://hub.docker.com/hardened-images/catalog). - -Choosing DHI offers the advantage of a production-ready image that is lightweight and secure. For more information, see [Docker Hardened Images](https://docs.docker.com/dhi/). - -{{< tabs >}} -{{< tab name="Using Docker Hardened Images" >}} - -Docker Hardened Images (DHIs) are available for Bun in the [Docker Hardened Images catalog](https://hub.docker.com/hardened-images/catalog/dhi/bun). You can pull DHIs directly from the `dhi.io` registry. - -1. Sign in to the DHI registry: - ```console - $ docker login dhi.io - ``` - -2. Pull the Bun DHI as `dhi.io/bun:1`. The tag (`1`) in this example refers to the version to the latest 1.x version of Bun. - - ```console - $ docker pull dhi.io/bun:1 - ``` - -For other available versions, refer to the [catalog](https://hub.docker.com/hardened-images/catalog/dhi/bun). - -```dockerfile -# Use the DHI Bun image as the base image -FROM dhi.io/bun:1 - -# Set the working directory in the container -WORKDIR /app - -# Copy the current directory contents into the container at /app -COPY . . - -# Expose the port on which the API will listen -EXPOSE 3000 - -# Run the server when the container launches -CMD ["bun", "server.js"] -``` - -{{< /tab >}} -{{< tab name="Using the official image" >}} - -Using the Docker Official Image is straightforward. In the following Dockerfile, you'll notice that the `FROM` instruction uses `oven/bun` as the base image. - -You can find the image on [Docker Hub](https://hub.docker.com/r/oven/bun). This is the Docker Official Image for Bun created by Oven, the company behind Bun, and it's available on Docker Hub. - -```dockerfile -# Use the official Bun image -FROM oven/bun:latest - -# Set the working directory in the container -WORKDIR /app - -# Copy the current directory contents into the container at /app -COPY . . - -# Expose the port on which the API will listen -EXPOSE 3000 - -# Run the server when the container launches -CMD ["bun", "server.js"] -``` - -{{< /tab >}} -{{< /tabs >}} - -In addition to specifying the base image, the Dockerfile also: - -- Sets the working directory in the container to `/app`. -- Copies the content of the current directory to the `/app` directory in the container. -- Exposes port 3000, where the API is listening for requests. -- And finally, starts the server when the container launches with the command `bun server.js`. - -## Run the application - -Inside the `bun-docker` directory, run the following command in a terminal. - -```console -$ docker compose up --build -``` - -Open a browser and view the application at [http://localhost:3000](http://localhost:3000). You will see a message `{"Status" : "OK"}` in the browser. - -In the terminal, press `ctrl`+`c` to stop the application. - -### Run the application in the background - -You can run the application detached from the terminal by adding the `-d` -option. Inside the `bun-docker` directory, run the following command -in a terminal. - -```console -$ docker compose up --build -d -``` - -Open a browser and view the application at [http://localhost:3000](http://localhost:3000). - - -In the terminal, run the following command to stop the application. - -```console -$ docker compose down -``` - -## Summary - -In this section, you learned how you can containerize and run your Bun -application using Docker. - -Related information: - - - [Dockerfile reference](/reference/dockerfile.md) - - [.dockerignore file](/reference/dockerfile.md#dockerignore-file) - - [Docker Compose overview](/manuals/compose/_index.md) - - [Compose file reference](/reference/compose-file/_index.md) - - [Docker Hardened Images](/dhi/) - -## Next steps - -In the next section, you'll learn how you can develop your application using -containers. diff --git a/content/guides/bun/deploy.md b/content/guides/bun/deploy.md deleted file mode 100644 index 61304b0e158b..000000000000 --- a/content/guides/bun/deploy.md +++ /dev/null @@ -1,141 +0,0 @@ ---- -title: Test your Bun deployment -linkTitle: Test your deployment -weight: 50 -keywords: deploy, kubernetes, bun -description: Learn how to develop locally using Kubernetes -aliases: -- /language/bun/deploy/ ---- - -## Prerequisites - -- Complete all the previous sections of this guide, starting with [Containerize a Bun application](containerize.md). -- [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. - -## Overview - -In this section, you'll learn how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. This allows you to test and debug your workloads on Kubernetes locally before deploying. - -## Create a Kubernetes YAML file - -In your `bun-docker` directory, create a file named -`docker-kubernetes.yml`. Open the file in an IDE or text editor and add -the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker -username and the name of the repository that you created in [Configure CI/CD for -your Bun application](configure-ci-cd.md). - -```yaml -apiVersion: apps/v1 -kind: Deployment -metadata: - name: docker-bun-demo - namespace: default -spec: - replicas: 1 - selector: - matchLabels: - app: bun-api - template: - metadata: - labels: - app: bun-api - spec: - containers: - - name: bun-api - image: DOCKER_USERNAME/REPO_NAME - imagePullPolicy: Always ---- -apiVersion: v1 -kind: Service -metadata: - name: service-entrypoint - namespace: default -spec: - type: NodePort - selector: - app: bun-api - ports: - - port: 3000 - targetPort: 3000 - nodePort: 30001 -``` - -In this Kubernetes YAML file, there are two objects, separated by the `---`: - - - A Deployment, describing a scalable group of identical pods. In this case, - you'll get just one replica, or copy of your pod. That pod, which is - described under `template`, has just one container in it. The - container is created from the image built by GitHub Actions in [Configure CI/CD for - your Bun application](configure-ci-cd.md). - - A NodePort service, which will route traffic from port 30001 on your host to - port 3000 inside the pods it routes to, allowing you to reach your app - from the network. - -To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). - -## Deploy and check your application - -1. In a terminal, navigate to `bun-docker` and deploy your application to - Kubernetes. - - ```console - $ kubectl apply -f docker-kubernetes.yml - ``` - - You should see output that looks like the following, indicating your Kubernetes objects were created successfully. - - ```text - deployment.apps/docker-bun-demo created - service/service-entrypoint created - ``` - -2. Make sure everything worked by listing your deployments. - - ```console - $ kubectl get deployments - ``` - - Your deployment should be listed as follows: - - ```shell - NAME READY UP-TO-DATE AVAILABLE AGE - docker-bun-demo 1/1 1 1 10s - ``` - - This indicates all one of the pods you asked for in your YAML are up and running. Do the same check for your services. - - ```console - $ kubectl get services - ``` - - You should get output like the following. - - ```shell - NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE - kubernetes ClusterIP 10.96.0.1 443/TCP 88m - service-entrypoint NodePort 10.105.145.223 3000:30001/TCP 83s - ``` - - In addition to the default `kubernetes` service, you can see your `service-entrypoint` service, accepting traffic on port 30001/TCP. - -3. In a browser, visit the following address. You should see the message `{"Status" : "OK"}`. - - ```console - http://localhost:30001/ - ``` - -4. Run the following command to tear down your application. - - ```console - $ kubectl delete -f docker-kubernetes.yml - ``` - -## Summary - -In this section, you learned how to use Docker Desktop to deploy your Bun application to a fully-featured Kubernetes environment on your development machine. - -Related information: - - [Kubernetes documentation](https://kubernetes.io/docs/home/) - - [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) - - [Swarm mode overview](/manuals/engine/swarm/_index.md) diff --git a/content/guides/bun/develop.md b/content/guides/bun/develop.md deleted file mode 100644 index 7de9bd63ed7e..000000000000 --- a/content/guides/bun/develop.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -title: Use containers for Bun development -linkTitle: Develop your app -weight: 20 -keywords: bun, local, development -description: Learn how to develop your Bun application locally. -aliases: -- /language/bun/develop/ ---- - -## Prerequisites - -Complete [Containerize a Bun application](containerize.md). - -## Overview - -In this section, you'll learn how to set up a development environment for your containerized application. This includes: - -- Configuring Compose to automatically update your running Compose services as you edit and save your code - -## Get the sample application - -Clone the sample application to use with this guide. Open a terminal, change directory to a directory that you want to work in, and run the following command to clone the repository: - -```console -$ git clone https://github.com/dockersamples/bun-docker.git && cd bun-docker -``` - -## Automatically update services - -Use Compose Watch to automatically update your running Compose services as you -edit and save your code. For more details about Compose Watch, see [Use Compose -Watch](/manuals/compose/how-tos/file-watch.md). - -Open your `compose.yml` file in an IDE or text editor and then add the Compose Watch instructions. The following example shows how to add Compose Watch to your `compose.yml` file. - -```yaml {hl_lines="9-12",linenos=true} -services: - server: - image: bun-server - build: - context: . - dockerfile: Dockerfile - ports: - - "3000:3000" - develop: - watch: - - action: rebuild - path: . -``` - -Run the following command to run your application with Compose Watch. - -```console -$ docker compose watch -``` - -Now, if you modify your `server.js` you will see the changes in real time without re-building the image. - -To test it out, open the `server.js` file in your favorite text editor and change the message from `{"Status" : "OK"}` to `{"Status" : "Updated"}`. Save the file and refresh your browser at `http://localhost:3000`. You should see the updated message. - -Press `ctrl+c` in the terminal to stop your application. - -## Summary - -In this section, you also learned how to use Compose Watch to automatically rebuild and run your container when you update your code. - -Related information: - - [Compose file reference](/reference/compose-file/) - - [Compose file watch](/manuals/compose/how-tos/file-watch.md) - - [Multi-stage builds](/manuals/build/building/multi-stage.md) - -## Next steps - -In the next section, you'll take a look at how to set up a CI/CD pipeline using GitHub Actions. diff --git a/content/guides/claude-code-model-runner.md b/content/guides/claude-code-model-runner.md index 652e74d3a1ec..8ac867e70ea4 100644 --- a/content/guides/claude-code-model-runner.md +++ b/content/guides/claude-code-model-runner.md @@ -5,8 +5,8 @@ summary: | Connect Claude Code to Docker Model Runner with the Anthropic-compatible API, package `gpt-oss` with a larger context window, and inspect requests. keywords: ai, claude code, docker model runner, anthropic, local models, coding assistant -tags: [ai] params: + tags: [ai] time: 10 minutes --- diff --git a/content/guides/claude-code-sandbox-model-runner.md b/content/guides/claude-code-sandbox-model-runner.md index 60d1fe0a312e..823a58593ee2 100644 --- a/content/guides/claude-code-sandbox-model-runner.md +++ b/content/guides/claude-code-sandbox-model-runner.md @@ -6,8 +6,8 @@ summary: | isolated microVM that talks to a local model on your host through the Anthropic-compatible API. keywords: ai, claude code, docker model runner, docker sandboxes, sbx, anthropic, local models, coding assistant -tags: [ai] params: + tags: [ai] time: 15 minutes --- diff --git a/content/guides/compose-bake/index.md b/content/guides/compose-bake.md similarity index 98% rename from content/guides/compose-bake/index.md rename to content/guides/compose-bake.md index c94f5a93082f..cad2cdfc1566 100644 --- a/content/guides/compose-bake/index.md +++ b/content/guides/compose-bake.md @@ -4,9 +4,8 @@ description: Learn how to build Docker Compose projects with Docker Buildx Bake summary: | This guide demonstrates how you can use Bake to build production-grade images for Docker Compose projects. keywords: docker compose, bake, buildx, multi-service, production builds, build configuration -languages: [] -tags: [devops] params: + tags: [cicd] time: 20 minutes --- @@ -211,7 +210,7 @@ $ docker buildx bake Start by redefining the default build group that Bake executes. The current default group includes a `seed` target — a Compose service used solely to populate the database with mock data. Since this target doesn't produce a -production image, it doesn’t need to be included in the build group. +production image, it doesn't need to be included in the build group. To customize the build configuration that Bake uses, create a new file at the root of the repository, alongside your `compose.yaml` file, named @@ -399,5 +398,5 @@ For more information about how to use Bake, check out these resources: - [Bake documentation](/manuals/build/bake/_index.md) - [Building with Bake from a Compose file](/manuals/build/bake/compose-file.md) - [Bake file reference](/manuals/build/bake/reference.md) -- [Mastering multi-platform builds, testing, and more with Docker Buildx Bake](/guides/bake/index.md) +- [Mastering multi-platform builds, testing, and more with Docker Buildx Bake](/guides/bake/) - [Bake GitHub Action](https://github.com/docker/bake-action) diff --git a/content/guides/container-supported-development.md b/content/guides/container-supported-development.md index 2362df2f3d8c..80dc6dc36e80 100644 --- a/content/guides/container-supported-development.md +++ b/content/guides/container-supported-development.md @@ -6,12 +6,11 @@ summary: | description: | Use containers in your local development loop to develop and test faster… even if your main app isn't running in containers. keywords: containers, local development, dependent services, testing, debugging, development environment -tags: [app-dev] params: + tags: [cicd] image: images/learning-paths/container-supported-development.png time: 20 minutes - resource_links: [] ---- + resource_links: []--- Containers offer a consistent way to build, share, and run applications across different environments. While containers are typically used to containerize your application, they also make it incredibly easy to run essential services needed for development. Instead of installing or connecting to a remote database, you can easily launch your own database. But the possibilities don't stop there. diff --git a/content/guides/cpp/_index.md b/content/guides/cpp/_index.md index 85f317765e3c..16bf60e9101e 100644 --- a/content/guides/cpp/_index.md +++ b/content/guides/cpp/_index.md @@ -5,16 +5,25 @@ description: Containerize and develop C++ applications using Docker. keywords: getting started, c++ summary: | This guide explains how to containerize C++ applications using Docker. -toc_min: 1 -toc_max: 2 aliases: - /language/cpp/ - /guides/language/cpp/ -languages: [cpp] + - /language/cpp/containerize/ + - /language/cpp/develop/ + - /language/cpp/configure-ci-cd/ + - /language/cpp/deploy/ + - /guides/cpp/configure-ci-cd/ + - /guides/cpp/containerize/ + - /guides/cpp/deploy/ + - /guides/cpp/develop/ + - /guides/cpp/multistage/ + - /guides/cpp/security/ params: + tags: [cicd] time: 20 minutes --- + The C++ getting started guide teaches you how to create a containerized C++ application using Docker. In this guide, you'll learn how to: > **Acknowledgment** @@ -31,3 +40,597 @@ The C++ getting started guide teaches you how to create a containerized C++ appl After completing the C++ getting started modules, you should be able to containerize your own C++ application based on the examples and instructions provided in this guide. Start by containerizing an existing C++ application. + +## Create a multi-stage build for your C++ application + +### Prerequisites + +- You have a [Git client](https://git-scm.com/downloads). The examples in this section use a command-line based Git client, but you can use any client. + +### Overview + +This section walks you through creating a multi-stage Docker build for a C++ application. +A multi-stage build is a Docker feature that allows you to use different base images for different stages of the build process, +so you can optimize the size of your final image and separate build dependencies from runtime dependencies. + +The standard practice for compiled languages like C++ is to have a build stage that compiles the code and a runtime stage that runs the compiled binary, +because the build dependencies are not needed at runtime. + +### Get the sample application + +Let's use a simple C++ application that prints `Hello, World!` to the terminal. To do so, clone the sample repository to use with this guide: + +```bash +$ git clone https://github.com/dockersamples/c-plus-plus-docker.git +``` + +The example for this section is under the `hello` directory in the repository. Get inside it and take a look at the files: + +```bash +$ cd c-plus-plus-docker/hello +$ ls +``` + +You should see the following files: + +```text +Dockerfile hello.cpp +``` + +### Check the Dockerfile + +Open the `Dockerfile` in an IDE or text editor. The `Dockerfile` contains the instructions for building the Docker image. + +```Dockerfile +# Stage 1: Build stage +FROM ubuntu:latest AS build + +# Install build-essential for compiling C++ code +RUN apt-get update && apt-get install -y build-essential + +# Set the working directory +WORKDIR /app + +# Copy the source code into the container +COPY hello.cpp . + +# Compile the C++ code statically to ensure it doesn't depend on runtime libraries +RUN g++ -o hello hello.cpp -static + +# Stage 2: Runtime stage +FROM scratch + +# Copy the static binary from the build stage +COPY --from=build /app/hello /hello + +# Command to run the binary +CMD ["/hello"] +``` + +The `Dockerfile` has two stages: + +1. **Build stage**: This stage uses the `ubuntu:latest` image to compile the C++ code and create a static binary. +2. **Runtime stage**: This stage uses the `scratch` image, which is an empty image, to copy the static binary from the build stage and run it. + +### Build the Docker image + +To build the Docker image, run the following command in the `hello` directory: + +```bash +$ docker build -t hello . +``` + +The `-t` flag tags the image with the name `hello`. + +### Run the Docker container + +To run the Docker container, use the following command: + +```bash +$ docker run hello +``` + +You should see the output `Hello, World!` in the terminal. + +### Summary + +In this section, you learned how to create a multi-stage build for a C++ application. Multi-stage builds help you optimize the size of your final image and separate build dependencies from runtime dependencies. +In this example, the final image only contains the static binary and doesn't include any build dependencies. + +As the image has an empty base, the usual OS tools are also absent. So, for example, you can't run a simple `ls` command in the container: + +```bash +$ docker run hello ls +``` + +This makes the image very lightweight and secure. + +## Containerize a C++ application + +### Prerequisites + +- You have a [Git client](https://git-scm.com/downloads). The examples in this section use a command-line based Git client, but you can use any client. + +### Overview + +This section walks you through containerizing and running a C++ application, using Docker Compose. + +### Get the sample application + +We're using the same sample repository that you used in the previous sections of this guide. If you haven't already cloned the repository, clone it now: + +```console +$ git clone https://github.com/dockersamples/c-plus-plus-docker.git +``` + +You should now have the following contents in your `c-plus-plus-docker` (root) +directory. + +```text +├── c-plus-plus-docker/ +│ ├── compose.yml +│ ├── Dockerfile +│ ├── LICENSE +│ ├── ok_api.cpp +│ └── README.md + +``` + +To learn more about the files in the repository, see the following: + +- [Dockerfile](/reference/dockerfile.md) +- [.dockerignore](/reference/dockerfile.md#dockerignore-file) +- [compose.yml](/reference/compose-file/_index.md) + +### Run the application + +Inside the `c-plus-plus-docker` directory, run the following command in a +terminal. + +```console +$ docker compose up --build +``` + +Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You will see a message `{"Status" : "OK"}` in the browser. + +In the terminal, press `ctrl`+`c` to stop the application. + +#### Run the application in the background + +You can run the application detached from the terminal by adding the `-d` +option. Inside the `c-plus-plus-docker` directory, run the following command +in a terminal. + +```console +$ docker compose up --build -d +``` + +Open a browser and view the application at [http://localhost:8080](http://localhost:8080). + +In the terminal, run the following command to stop the application. + +```console +$ docker compose down +``` + +For more information about Compose commands, see the [Compose CLI +reference](/reference/cli/docker/compose/). + +### Summary + +In this section, you learned how you can containerize and run your C++ +application using Docker. + +Related information: + +- [Docker Compose overview](/manuals/compose/_index.md) + +### Next steps + +In the next section, you'll learn how you can develop your application using +containers. + +## Use containers for C++ development + +### Prerequisites + +Complete [Containerize a C++ application](containerize.md). + +### Overview + +In this section, you'll learn how to set up a development environment for your containerized application. This includes: + +- Configuring Compose to automatically update your running Compose services as you edit and save your code + +### Get the sample application + +Clone the sample application to use with this guide. Open a terminal, change directory to a directory that you want to work in, and run the following command to clone the repository: + +```console +$ git clone https://github.com/dockersamples/c-plus-plus-docker.git && cd c-plus-plus-docker +``` + +### Automatically update services + +Use Compose Watch to automatically update your running Compose services as you +edit and save your code. For more details about Compose Watch, see [Use Compose +Watch](/manuals/compose/how-tos/file-watch.md). + +Open your `compose.yml` file in an IDE or text editor and then add the Compose Watch instructions. The following example shows how to add Compose Watch to your `compose.yml` file. + +```yaml {hl_lines="11-14",linenos=true} +services: + ok-api: + image: ok-api + build: + context: . + dockerfile: Dockerfile + ports: + - "8080:8080" + develop: + watch: + - action: rebuild + path: . +``` + +Run the following command to run your application with Compose Watch. + +```console +$ docker compose watch +``` + +Now, if you modify your `ok_api.cpp` you will see the changes in real time without re-building the image. + +To test it out, open the `ok_api.cpp` file in your favorite text editor and change the message from `{"Status" : "OK"}` to `{"Status" : "Updated"}`. Save the file and refresh your browser at [http://localhost:8080](http://localhost:8080). You should see the updated message. + +Press `ctrl+c` in the terminal to stop your application. + +### Summary + +In this section, you also learned how to use Compose Watch to automatically rebuild and run your container when you update your code. + +Related information: + +- [Compose file reference](/reference/compose-file/) +- [Compose file watch](/manuals/compose/how-tos/file-watch.md) +- [Multi-stage builds](/manuals/build/building/multi-stage.md) + +### Next steps + +In the next section, you'll take a look at how to set up a CI/CD pipeline using GitHub Actions. + +## Configure CI/CD for your C++ application + +### Prerequisites + +Complete all the previous sections of this guide, starting with [Containerize a C++ application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. + +### Overview + +In this section, you'll learn how to set up and use GitHub Actions to build and test your Docker image as well as push it to Docker Hub. You will complete the following steps: + +1. Create a new repository on GitHub. +2. Define the GitHub Actions workflow. +3. Run the workflow. + +### Step one: Create the repository + +Create a GitHub repository, configure the Docker Hub credentials, and push your source code. + +1. [Create a new repository](https://github.com/new) on GitHub. + +2. Open the repository **Settings**, and go to **Secrets and variables** > + **Actions**. + +3. Create a new **Repository variable** named `DOCKER_USERNAME` and your Docker ID as a value. + +4. Create a new [Personal Access Token (PAT)](/manuals/security/access-tokens.md#create-an-access-token) for Docker Hub. You can name this token `docker-tutorial`. Make sure access permissions include Read and Write. + +5. Add the PAT as a **Repository secret** in your GitHub repository, with the name + `DOCKERHUB_TOKEN`. + +6. In your local repository on your machine, run the following command to change + the origin to the repository you just created. Make sure you change + `your-username` to your GitHub username and `your-repository` to the name of + the repository you created. + + ```console + $ git remote set-url origin https://github.com/your-username/your-repository.git + ``` + +7. Run the following commands to stage, commit, and push your local repository to GitHub. + + ```console + $ git add -A + $ git commit -m "my commit" + $ git push -u origin main + ``` + +### Step two: Set up the workflow + +Set up your GitHub Actions workflow for building, testing, and pushing the image +to Docker Hub. + +1. Go to your repository on GitHub and then select the **Actions** tab. + +2. Select **set up a workflow yourself**. + + This takes you to a page for creating a new GitHub actions workflow file in + your repository, under `.github/workflows/main.yml` by default. + +3. In the editor window, copy and paste the following YAML configuration and commit the changes. + + ```yaml + name: ci + + on: + push: + branches: + - main + + jobs: + build: + runs-on: ubuntu-latest + steps: + - name: Login to Docker Hub + uses: docker/login-action@{{% param "login_action_version" %}} + with: + username: ${{ vars.DOCKER_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} + + - name: Build and push + uses: docker/build-push-action@{{% param "build_push_action_version" %}} + with: + platforms: linux/amd64,linux/arm64 + push: true + tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest + ``` + + For more information about the YAML syntax for `docker/build-push-action`, + refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). + +### Step three: Run the workflow + +Save the workflow file and run the job. + +1. Select **Commit changes...** and push the changes to the `main` branch. + + After pushing the commit, the workflow starts automatically. + +2. Go to the **Actions** tab. It displays the workflow. + + Selecting the workflow shows you the breakdown of all the steps. + +3. When the workflow is complete, go to your + [repositories on Docker Hub](https://hub.docker.com/repositories). + + If you see the new repository in that list, it means the GitHub Actions + successfully pushed the image to Docker Hub. + +### Summary + +In this section, you learned how to set up a GitHub Actions workflow for your C++ application. + +Related information: + +- [Introduction to GitHub Actions](/guides/gha.md) +- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) +- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) + +### Next steps + +Next, learn how you can locally test and debug your workloads on Kubernetes before deploying. + +## Test your C++ deployment + +### Prerequisites + +- Complete all the previous sections of this guide, starting with [Containerize a C++ application](containerize.md). +- [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. + +### Overview + +In this section, you'll learn how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. This allows you to test and debug your workloads on Kubernetes locally before deploying. + +### Create a Kubernetes YAML file + +In your `c-plus-plus-docker` directory, create a file named +`docker-kubernetes.yml`. Open the file in an IDE or text editor and add +the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker +username and the name of the repository that you created in [Configure CI/CD for +your C++ application](configure-ci-cd.md). + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: docker-c-plus-plus-demo + namespace: default +spec: + replicas: 1 + selector: + matchLabels: + service: ok-api + template: + metadata: + labels: + service: ok-api + spec: + containers: + - name: ok-api-service + image: DOCKER_USERNAME/REPO_NAME + imagePullPolicy: Always +--- +apiVersion: v1 +kind: Service +metadata: + name: service-entrypoint + namespace: default +spec: + type: NodePort + selector: + service: ok-api + ports: + - port: 8080 + targetPort: 8080 + nodePort: 30001 +``` + +In this Kubernetes YAML file, there are two objects, separated by the `---`: + +- A Deployment, describing a scalable group of identical pods. In this case, + you'll get just one replica, or copy of your pod. That pod, which is + described under `template`, has just one container in it. The + container is created from the image built by GitHub Actions in [Configure CI/CD for + your C++ application](configure-ci-cd.md). +- A NodePort service, which will route traffic from port 30001 on your host to + port 8080 inside the pods it routes to, allowing you to reach your app + from the network. + +To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). + +### Deploy and check your application + +1. In a terminal, navigate to `c-plus-plus-docker` and deploy your application to + Kubernetes. + + ```console + $ kubectl apply -f docker-kubernetes.yml + ``` + + You should see output that looks like the following, indicating your Kubernetes objects were created successfully. + + ```text + deployment.apps/docker-c-plus-plus-demo created + service/service-entrypoint created + ``` + +2. Make sure everything worked by listing your deployments. + + ```console + $ kubectl get deployments + ``` + + Your deployment should be listed as follows: + + ```shell + NAME READY UP-TO-DATE AVAILABLE AGE + docker-c-plus-plus-demo 1/1 1 1 10s + ``` + + This indicates all one of the pods you asked for in your YAML are up and running. Do the same check for your services. + + ```console + $ kubectl get services + ``` + + You should get output like the following. + + ```shell + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + kubernetes ClusterIP 10.96.0.1 443/TCP 88m + service-entrypoint NodePort 10.105.145.223 8080:30001/TCP 83s + ``` + + In addition to the default `kubernetes` service, you can see your `service-entrypoint` service, accepting traffic on port 30001/TCP. + +3. In a browser, visit the following address. You should see the message `{"Status" : "OK"}`. + + ```console + http://localhost:30001/ + ``` + +4. Run the following command to tear down your application. + + ```console + $ kubectl delete -f docker-kubernetes.yml + ``` + +### Summary + +In this section, you learned how to use Docker Desktop to deploy your C++ application to a fully-featured Kubernetes environment on your development machine. + +Related information: + +- [Kubernetes documentation](https://kubernetes.io/docs/home/) +- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) +- [Swarm mode overview](/manuals/engine/swarm/_index.md) + +## Supply-chain security for C++ Docker images + +### Prerequisites + +- You have a [Git client](https://git-scm.com/downloads). The examples in this section use a command-line based Git client, but you can use any client. +- You have a Docker Desktop installed, with containerd enabled for pulling and storing images (it's a checkbox in **Settings** > **General**). Otherwise, if you use Docker Engine: + - You have the [Docker Scout CLI plugin](https://docs.docker.com/scout/install/) installed. To install it on Docker Engine, use the following command: + + ```bash + $ curl -sSfL https://raw.githubusercontent.com/docker/scout-cli/main/install.sh | sh -s -- + ``` + + - You have [containerd enabled](https://docs.docker.com/engine/storage/containerd/) for Docker Engine. + +### Overview + +This section walks you through extracting Software Bill of Materials (SBOMs) from a C++ Docker image using Docker Scout. SBOMs provide a detailed list of all the components in a software package, including their versions and licenses. You can use SBOMs to track the provenance of your software and ensure that it complies with your organization's security and licensing policies. + +### Generate an SBOM + +Here we will use the Docker image that we built in the [Create a multi-stage build for your C++ application](/guides/language/cpp/multistage/) guide. If you haven't already built the image, follow the steps in that guide to build the image. +The image is named `hello`. To generate an SBOM for the `hello` image, run the following command: + +```bash +$ docker scout sbom --format list hello +``` + +The command will say "No packages discovered". This is because the final image is a scratch image and doesn't have any packages. + +### Generate an SBOM attestation + +The SBOM can be generated during the build process and "attached" to the image. This is called an SBOM attestation. +To generate an SBOM attestation for the `hello` image, first let's change the Dockerfile: + +```Dockerfile +ARG BUILDKIT_SBOM_SCAN_STAGE=true + +FROM ubuntu:latest AS build + +RUN apt-get update && apt-get install -y build-essential + +WORKDIR /app + +COPY hello.cpp . + +RUN g++ -o hello hello.cpp -static + +# -------------------- +FROM scratch + +COPY --from=build /app/hello /hello + +CMD ["/hello"] +``` + +The first line `ARG BUILDKIT_SBOM_SCAN_STAGE=true` enables SBOM scanning in the build stage. +Now, build the image with the following command: + +```bash +$ docker buildx build --sbom=true -t hello:sbom . +``` + +This command will build the image and generate an SBOM attestation. You can verify that the SBOM is attached to the image by running the following command: + +```bash +$ docker scout sbom --format list hello:sbom +``` + +Docker Scout reads the SBOM attestation when one is available, so this command reports packages from the build-stage metadata instead of indexing only the final scratch image filesystem. + +### Summary + +In this section, you learned how to generate SBOM attestation for a C++ Docker image during the build process. +Image scanners that inspect only the final filesystem may not identify packages in scratch images. +Use SBOM attestations to preserve package metadata from the build. diff --git a/content/guides/cpp/configure-ci-cd.md b/content/guides/cpp/configure-ci-cd.md deleted file mode 100644 index e2992f2243b0..000000000000 --- a/content/guides/cpp/configure-ci-cd.md +++ /dev/null @@ -1,133 +0,0 @@ ---- -title: Configure CI/CD for your C++ application -linkTitle: Configure CI/CD -weight: 40 -keywords: ci/cd, github actions, c++, shiny -description: Learn how to configure CI/CD using GitHub Actions for your C++ application. -aliases: - - /language/cpp/configure-ci-cd/ - - /guides/language/cpp/configure-ci-cd/ ---- - -## Prerequisites - -Complete all the previous sections of this guide, starting with [Containerize a C++ application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. - -## Overview - -In this section, you'll learn how to set up and use GitHub Actions to build and test your Docker image as well as push it to Docker Hub. You will complete the following steps: - -1. Create a new repository on GitHub. -2. Define the GitHub Actions workflow. -3. Run the workflow. - -## Step one: Create the repository - -Create a GitHub repository, configure the Docker Hub credentials, and push your source code. - -1. [Create a new repository](https://github.com/new) on GitHub. - -2. Open the repository **Settings**, and go to **Secrets and variables** > - **Actions**. - -3. Create a new **Repository variable** named `DOCKER_USERNAME` and your Docker ID as a value. - -4. Create a new [Personal Access Token (PAT)](/manuals/security/access-tokens.md#create-an-access-token) for Docker Hub. You can name this token `docker-tutorial`. Make sure access permissions include Read and Write. - -5. Add the PAT as a **Repository secret** in your GitHub repository, with the name - `DOCKERHUB_TOKEN`. - -6. In your local repository on your machine, run the following command to change - the origin to the repository you just created. Make sure you change - `your-username` to your GitHub username and `your-repository` to the name of - the repository you created. - - ```console - $ git remote set-url origin https://github.com/your-username/your-repository.git - ``` - -7. Run the following commands to stage, commit, and push your local repository to GitHub. - - ```console - $ git add -A - $ git commit -m "my commit" - $ git push -u origin main - ``` - -## Step two: Set up the workflow - -Set up your GitHub Actions workflow for building, testing, and pushing the image -to Docker Hub. - -1. Go to your repository on GitHub and then select the **Actions** tab. - -2. Select **set up a workflow yourself**. - - This takes you to a page for creating a new GitHub actions workflow file in - your repository, under `.github/workflows/main.yml` by default. - -3. In the editor window, copy and paste the following YAML configuration and commit the changes. - - ```yaml - name: ci - - on: - push: - branches: - - main - - jobs: - build: - runs-on: ubuntu-latest - steps: - - name: Login to Docker Hub - uses: docker/login-action@{{% param "login_action_version" %}} - with: - username: ${{ vars.DOCKER_USERNAME }} - password: ${{ secrets.DOCKERHUB_TOKEN }} - - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} - - - name: Build and push - uses: docker/build-push-action@{{% param "build_push_action_version" %}} - with: - platforms: linux/amd64,linux/arm64 - push: true - tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest - ``` - - For more information about the YAML syntax for `docker/build-push-action`, - refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). - -## Step three: Run the workflow - -Save the workflow file and run the job. - -1. Select **Commit changes...** and push the changes to the `main` branch. - - After pushing the commit, the workflow starts automatically. - -2. Go to the **Actions** tab. It displays the workflow. - - Selecting the workflow shows you the breakdown of all the steps. - -3. When the workflow is complete, go to your - [repositories on Docker Hub](https://hub.docker.com/repositories). - - If you see the new repository in that list, it means the GitHub Actions - successfully pushed the image to Docker Hub. - -## Summary - -In this section, you learned how to set up a GitHub Actions workflow for your C++ application. - -Related information: - -- [Introduction to GitHub Actions](/guides/gha.md) -- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) -- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) - -## Next steps - -Next, learn how you can locally test and debug your workloads on Kubernetes before deploying. diff --git a/content/guides/cpp/containerize.md b/content/guides/cpp/containerize.md deleted file mode 100644 index 974ff7d3ed86..000000000000 --- a/content/guides/cpp/containerize.md +++ /dev/null @@ -1,93 +0,0 @@ ---- -title: Containerize a C++ application -linkTitle: Containerize -weight: 10 -keywords: C++, containerize, initialize -description: Learn how to use Docker Compose to build and run a C++ application. -aliases: - - /language/cpp/containerize/ - - /guides/language/cpp/containerize/ ---- - -## Prerequisites - -- You have a [Git client](https://git-scm.com/downloads). The examples in this section use a command-line based Git client, but you can use any client. - -## Overview - -This section walks you through containerizing and running a C++ application, using Docker Compose. - -## Get the sample application - -We're using the same sample repository that you used in the previous sections of this guide. If you haven't already cloned the repository, clone it now: - -```console -$ git clone https://github.com/dockersamples/c-plus-plus-docker.git -``` - -You should now have the following contents in your `c-plus-plus-docker` (root) -directory. - -```text -├── c-plus-plus-docker/ -│ ├── compose.yml -│ ├── Dockerfile -│ ├── LICENSE -│ ├── ok_api.cpp -│ └── README.md - -``` - -To learn more about the files in the repository, see the following: - -- [Dockerfile](/reference/dockerfile.md) -- [.dockerignore](/reference/dockerfile.md#dockerignore-file) -- [compose.yml](/reference/compose-file/_index.md) - -## Run the application - -Inside the `c-plus-plus-docker` directory, run the following command in a -terminal. - -```console -$ docker compose up --build -``` - -Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You will see a message `{"Status" : "OK"}` in the browser. - -In the terminal, press `ctrl`+`c` to stop the application. - -### Run the application in the background - -You can run the application detached from the terminal by adding the `-d` -option. Inside the `c-plus-plus-docker` directory, run the following command -in a terminal. - -```console -$ docker compose up --build -d -``` - -Open a browser and view the application at [http://localhost:8080](http://localhost:8080). - -In the terminal, run the following command to stop the application. - -```console -$ docker compose down -``` - -For more information about Compose commands, see the [Compose CLI -reference](/reference/cli/docker/compose/). - -## Summary - -In this section, you learned how you can containerize and run your C++ -application using Docker. - -Related information: - -- [Docker Compose overview](/manuals/compose/_index.md) - -## Next steps - -In the next section, you'll learn how you can develop your application using -containers. diff --git a/content/guides/cpp/deploy.md b/content/guides/cpp/deploy.md deleted file mode 100644 index 120067302ba1..000000000000 --- a/content/guides/cpp/deploy.md +++ /dev/null @@ -1,143 +0,0 @@ ---- -title: Test your C++ deployment -linkTitle: Test your deployment -weight: 50 -keywords: deploy, kubernetes, c++ -description: Learn how to develop locally using Kubernetes -aliases: - - /language/cpp/deploy/ - - /guides/language/cpp/deploy/ ---- - -## Prerequisites - -- Complete all the previous sections of this guide, starting with [Containerize a C++ application](containerize.md). -- [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. - -## Overview - -In this section, you'll learn how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. This allows you to test and debug your workloads on Kubernetes locally before deploying. - -## Create a Kubernetes YAML file - -In your `c-plus-plus-docker` directory, create a file named -`docker-kubernetes.yml`. Open the file in an IDE or text editor and add -the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker -username and the name of the repository that you created in [Configure CI/CD for -your C++ application](configure-ci-cd.md). - -```yaml -apiVersion: apps/v1 -kind: Deployment -metadata: - name: docker-c-plus-plus-demo - namespace: default -spec: - replicas: 1 - selector: - matchLabels: - service: ok-api - template: - metadata: - labels: - service: ok-api - spec: - containers: - - name: ok-api-service - image: DOCKER_USERNAME/REPO_NAME - imagePullPolicy: Always ---- -apiVersion: v1 -kind: Service -metadata: - name: service-entrypoint - namespace: default -spec: - type: NodePort - selector: - service: ok-api - ports: - - port: 8080 - targetPort: 8080 - nodePort: 30001 -``` - -In this Kubernetes YAML file, there are two objects, separated by the `---`: - -- A Deployment, describing a scalable group of identical pods. In this case, - you'll get just one replica, or copy of your pod. That pod, which is - described under `template`, has just one container in it. The - container is created from the image built by GitHub Actions in [Configure CI/CD for - your C++ application](configure-ci-cd.md). -- A NodePort service, which will route traffic from port 30001 on your host to - port 8080 inside the pods it routes to, allowing you to reach your app - from the network. - -To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). - -## Deploy and check your application - -1. In a terminal, navigate to `c-plus-plus-docker` and deploy your application to - Kubernetes. - - ```console - $ kubectl apply -f docker-kubernetes.yml - ``` - - You should see output that looks like the following, indicating your Kubernetes objects were created successfully. - - ```text - deployment.apps/docker-c-plus-plus-demo created - service/service-entrypoint created - ``` - -2. Make sure everything worked by listing your deployments. - - ```console - $ kubectl get deployments - ``` - - Your deployment should be listed as follows: - - ```shell - NAME READY UP-TO-DATE AVAILABLE AGE - docker-c-plus-plus-demo 1/1 1 1 10s - ``` - - This indicates all one of the pods you asked for in your YAML are up and running. Do the same check for your services. - - ```console - $ kubectl get services - ``` - - You should get output like the following. - - ```shell - NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE - kubernetes ClusterIP 10.96.0.1 443/TCP 88m - service-entrypoint NodePort 10.105.145.223 8080:30001/TCP 83s - ``` - - In addition to the default `kubernetes` service, you can see your `service-entrypoint` service, accepting traffic on port 30001/TCP. - -3. In a browser, visit the following address. You should see the message `{"Status" : "OK"}`. - - ```console - http://localhost:30001/ - ``` - -4. Run the following command to tear down your application. - - ```console - $ kubectl delete -f docker-kubernetes.yml - ``` - -## Summary - -In this section, you learned how to use Docker Desktop to deploy your C++ application to a fully-featured Kubernetes environment on your development machine. - -Related information: - -- [Kubernetes documentation](https://kubernetes.io/docs/home/) -- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) -- [Swarm mode overview](/manuals/engine/swarm/_index.md) diff --git a/content/guides/cpp/develop.md b/content/guides/cpp/develop.md deleted file mode 100644 index 016eac56cefa..000000000000 --- a/content/guides/cpp/develop.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -title: Use containers for C++ development -linkTitle: Develop your app -weight: 20 -keywords: C++, local, development -description: Learn how to develop your C++ application locally. -aliases: - - /language/cpp/develop/ - - /guides/language/cpp/develop/ ---- - -## Prerequisites - -Complete [Containerize a C++ application](containerize.md). - -## Overview - -In this section, you'll learn how to set up a development environment for your containerized application. This includes: - -- Configuring Compose to automatically update your running Compose services as you edit and save your code - -## Get the sample application - -Clone the sample application to use with this guide. Open a terminal, change directory to a directory that you want to work in, and run the following command to clone the repository: - -```console -$ git clone https://github.com/dockersamples/c-plus-plus-docker.git && cd c-plus-plus-docker -``` - -## Automatically update services - -Use Compose Watch to automatically update your running Compose services as you -edit and save your code. For more details about Compose Watch, see [Use Compose -Watch](/manuals/compose/how-tos/file-watch.md). - -Open your `compose.yml` file in an IDE or text editor and then add the Compose Watch instructions. The following example shows how to add Compose Watch to your `compose.yml` file. - -```yaml {hl_lines="11-14",linenos=true} -services: - ok-api: - image: ok-api - build: - context: . - dockerfile: Dockerfile - ports: - - "8080:8080" - develop: - watch: - - action: rebuild - path: . -``` - -Run the following command to run your application with Compose Watch. - -```console -$ docker compose watch -``` - -Now, if you modify your `ok_api.cpp` you will see the changes in real time without re-building the image. - -To test it out, open the `ok_api.cpp` file in your favorite text editor and change the message from `{"Status" : "OK"}` to `{"Status" : "Updated"}`. Save the file and refresh your browser at [http://localhost:8080](http://localhost:8080). You should see the updated message. - -Press `ctrl+c` in the terminal to stop your application. - -## Summary - -In this section, you also learned how to use Compose Watch to automatically rebuild and run your container when you update your code. - -Related information: - -- [Compose file reference](/reference/compose-file/) -- [Compose file watch](/manuals/compose/how-tos/file-watch.md) -- [Multi-stage builds](/manuals/build/building/multi-stage.md) - -## Next steps - -In the next section, you'll take a look at how to set up a CI/CD pipeline using GitHub Actions. diff --git a/content/guides/cpp/multistage.md b/content/guides/cpp/multistage.md deleted file mode 100644 index 4a73d6f42f0a..000000000000 --- a/content/guides/cpp/multistage.md +++ /dev/null @@ -1,112 +0,0 @@ ---- -title: Create a multi-stage build for your C++ application -linkTitle: Multi-stage build -weight: 5 -keywords: C++, containerize, multi-stage -description: Learn how to create a multi-stage build for a C++ application. -aliases: -- /language/cpp/multistage/ -- /guides/language/cpp/multistage/ ---- - -## Prerequisites - -- You have a [Git client](https://git-scm.com/downloads). The examples in this section use a command-line based Git client, but you can use any client. - -## Overview - -This section walks you through creating a multi-stage Docker build for a C++ application. -A multi-stage build is a Docker feature that allows you to use different base images for different stages of the build process, -so you can optimize the size of your final image and separate build dependencies from runtime dependencies. - -The standard practice for compiled languages like C++ is to have a build stage that compiles the code and a runtime stage that runs the compiled binary, -because the build dependencies are not needed at runtime. - -## Get the sample application - -Let's use a simple C++ application that prints `Hello, World!` to the terminal. To do so, clone the sample repository to use with this guide: - -```bash -$ git clone https://github.com/dockersamples/c-plus-plus-docker.git -``` - -The example for this section is under the `hello` directory in the repository. Get inside it and take a look at the files: - -```bash -$ cd c-plus-plus-docker/hello -$ ls -``` - -You should see the following files: - -```text -Dockerfile hello.cpp -``` - -## Check the Dockerfile - -Open the `Dockerfile` in an IDE or text editor. The `Dockerfile` contains the instructions for building the Docker image. - -```Dockerfile -# Stage 1: Build stage -FROM ubuntu:latest AS build - -# Install build-essential for compiling C++ code -RUN apt-get update && apt-get install -y build-essential - -# Set the working directory -WORKDIR /app - -# Copy the source code into the container -COPY hello.cpp . - -# Compile the C++ code statically to ensure it doesn't depend on runtime libraries -RUN g++ -o hello hello.cpp -static - -# Stage 2: Runtime stage -FROM scratch - -# Copy the static binary from the build stage -COPY --from=build /app/hello /hello - -# Command to run the binary -CMD ["/hello"] -``` - -The `Dockerfile` has two stages: - -1. **Build stage**: This stage uses the `ubuntu:latest` image to compile the C++ code and create a static binary. -2. **Runtime stage**: This stage uses the `scratch` image, which is an empty image, to copy the static binary from the build stage and run it. - -## Build the Docker image - -To build the Docker image, run the following command in the `hello` directory: - -```bash -$ docker build -t hello . -``` - -The `-t` flag tags the image with the name `hello`. - -## Run the Docker container - -To run the Docker container, use the following command: - -```bash -$ docker run hello -``` - -You should see the output `Hello, World!` in the terminal. - -## Summary - -In this section, you learned how to create a multi-stage build for a C++ application. Multi-stage builds help you optimize the size of your final image and separate build dependencies from runtime dependencies. -In this example, the final image only contains the static binary and doesn't include any build dependencies. - -As the image has an empty base, the usual OS tools are also absent. So, for example, you can't run a simple `ls` command in the container: - -```bash -$ docker run hello ls -``` - -This makes the image very lightweight and secure. diff --git a/content/guides/cpp/security.md b/content/guides/cpp/security.md deleted file mode 100644 index 7318fb5af6d5..000000000000 --- a/content/guides/cpp/security.md +++ /dev/null @@ -1,84 +0,0 @@ ---- -title: Supply-chain security for C++ Docker images -linkTitle: Supply-chain security -weight: 60 -keywords: C++, security, multi-stage -description: Learn how to extract SBOMs from C++ Docker images. -aliases: -- /language/cpp/security/ -- /guides/language/cpp/security/ ---- - -## Prerequisites - -- You have a [Git client](https://git-scm.com/downloads). The examples in this section use a command-line based Git client, but you can use any client. -- You have a Docker Desktop installed, with containerd enabled for pulling and storing images (it's a checkbox in **Settings** > **General**). Otherwise, if you use Docker Engine: - - You have the [Docker Scout CLI plugin](https://docs.docker.com/scout/install/) installed. To install it on Docker Engine, use the following command: - - ```bash - $ curl -sSfL https://raw.githubusercontent.com/docker/scout-cli/main/install.sh | sh -s -- - ``` - - - You have [containerd enabled](https://docs.docker.com/engine/storage/containerd/) for Docker Engine. - -## Overview - -This section walks you through extracting Software Bill of Materials (SBOMs) from a C++ Docker image using Docker Scout. SBOMs provide a detailed list of all the components in a software package, including their versions and licenses. You can use SBOMs to track the provenance of your software and ensure that it complies with your organization's security and licensing policies. - -## Generate an SBOM - -Here we will use the Docker image that we built in the [Create a multi-stage build for your C++ application](/guides/language/cpp/multistage/) guide. If you haven't already built the image, follow the steps in that guide to build the image. -The image is named `hello`. To generate an SBOM for the `hello` image, run the following command: - -```bash -$ docker scout sbom --format list hello -``` - -The command will say "No packages discovered". This is because the final image is a scratch image and doesn't have any packages. - -## Generate an SBOM attestation - -The SBOM can be generated during the build process and "attached" to the image. This is called an SBOM attestation. -To generate an SBOM attestation for the `hello` image, first let's change the Dockerfile: - -```Dockerfile -ARG BUILDKIT_SBOM_SCAN_STAGE=true - -FROM ubuntu:latest AS build - -RUN apt-get update && apt-get install -y build-essential - -WORKDIR /app - -COPY hello.cpp . - -RUN g++ -o hello hello.cpp -static - -# -------------------- -FROM scratch - -COPY --from=build /app/hello /hello - -CMD ["/hello"] -``` - -The first line `ARG BUILDKIT_SBOM_SCAN_STAGE=true` enables SBOM scanning in the build stage. -Now, build the image with the following command: - -```bash -$ docker buildx build --sbom=true -t hello:sbom . -``` - -This command will build the image and generate an SBOM attestation. You can verify that the SBOM is attached to the image by running the following command: - -```bash -$ docker scout sbom --format list hello:sbom -``` - -Docker Scout reads the SBOM attestation when one is available, so this command reports packages from the build-stage metadata instead of indexing only the final scratch image filesystem. - -## Summary - -In this section, you learned how to generate SBOM attestation for a C++ Docker image during the build process. -Image scanners that inspect only the final filesystem may not identify packages in scratch images. -Use SBOM attestations to preserve package metadata from the build. diff --git a/content/guides/databases.md b/content/guides/databases.md index 8c2e2cd9e3a6..8bbcbdefe043 100644 --- a/content/guides/databases.md +++ b/content/guides/databases.md @@ -4,10 +4,10 @@ keywords: database, mysql title: Use containerized databases summary: | Learn how to effectively run and manage databases as containers. -tags: [databases] aliases: - /guides/use-case/databases/ params: + tags: [databases] time: 20 minutes --- diff --git a/content/guides/deno/_index.md b/content/guides/deno/_index.md index f2a9dc38b715..79686d78ed68 100644 --- a/content/guides/deno/_index.md +++ b/content/guides/deno/_index.md @@ -5,12 +5,17 @@ title: Deno language-specific guide summary: | Learn how to containerize JavaScript applications with the Deno runtime using Docker. linkTitle: Deno -languages: [js] -tags: [dhi] +aliases: + - /guides/deno/configure-ci-cd/ + - /guides/deno/containerize/ + - /guides/deno/deploy/ + - /guides/deno/develop/ params: + tags: [cicd] time: 10 minutes --- + The Deno getting started guide teaches you how to create a containerized Deno application using Docker. > **Acknowledgment** @@ -33,3 +38,519 @@ The Deno getting started guide teaches you how to create a containerized Deno ap After completing the Deno getting started modules, you should be able to containerize your own Deno application based on the examples and instructions provided in this guide. Start by containerizing an existing Deno application. + +## Containerize a Deno application + +### Prerequisites + +* You have a [Git client](https://git-scm.com/downloads). The examples in this section use a command-line based Git client, but you can use any client. + +### Overview + +For a long time, Node.js has been the go-to runtime for server-side JavaScript applications. However, recent years have introduced new alternative runtimes, including [Deno](https://deno.land/). Like Node.js, Deno is a JavaScript and TypeScript runtime, but it takes a fresh approach with modern security features, a built-in standard library, and native support for TypeScript. + +Why develop Deno applications with Docker? Having a choice of runtimes is exciting, but managing multiple runtimes and their dependencies consistently across environments can be tricky. This is where Docker proves invaluable. Using containers to create and destroy environments on demand simplifies runtime management and ensures consistency. Additionally, as Deno continues to grow and evolve, Docker helps establish a reliable and reproducible development environment, minimizing setup challenges and streamlining the workflow. + +### Get the sample application + +Clone the sample application to use with this guide. Open a terminal, change +directory to a directory that you want to work in, and run the following +command to clone the repository: + +```console +$ git clone https://github.com/dockersamples/docker-deno.git && cd docker-deno +``` + +You should now have the following contents in your `deno-docker` directory. + +```text +├── deno-docker/ +│ ├── compose.yml +│ ├── Dockerfile +│ ├── LICENSE +│ ├── server.ts +│ └── README.md +``` + +### Understand the sample application + +The sample application is a simple Deno application that uses the Oak framework to create a simple API that returns a JSON response. The application listens on port 8000 and returns a message `{"Status" : "OK"}` when you access the application in a browser. + +```typescript +// server.ts +import { Application, Router } from "https://deno.land/x/oak@v12.0.0/mod.ts"; + +const app = new Application(); +const router = new Router(); + +// Define a route that returns JSON +router.get("/", (context) => { + context.response.body = { Status: "OK" }; + context.response.type = "application/json"; +}); + +app.use(router.routes()); +app.use(router.allowedMethods()); + +console.log("Server running on http://localhost:8000"); +await app.listen({ port: 8000 }); +``` + +### Create a Dockerfile + +Before creating a Dockerfile, you need to choose a base image. You can either use the [Deno Docker Official Image](https://hub.docker.com/r/denoland/deno) or a Docker Hardened Image (DHI) from the [Hardened Image catalog](https://hub.docker.com/hardened-images/catalog). + +Choosing DHI offers the advantage of a production-ready image that is lightweight and secure. For more information, see [Docker Hardened Images](https://docs.docker.com/dhi/). + +{{< tabs >}} +{{< tab name="Using Docker Hardened Images" >}} + +Docker Hardened Images (DHIs) are available for Deno in the [Docker Hardened Images catalog](https://hub.docker.com/hardened-images/catalog/dhi/deno). You can pull DHIs directly from the `dhi.io` registry. + +1. Sign in to the DHI registry: + + ```console + $ docker login dhi.io + ``` + +2. Pull the Deno DHI as `dhi.io/deno:2`. The tag (`2`) in this example refers to the version to the latest 2.x version of Deno. + + ```console + $ docker pull dhi.io/deno:2 + ``` + +For other available versions, refer to the [catalog](https://hub.docker.com/hardened-images/catalog/dhi/deno). + +```dockerfile +# Use the DHI Deno image as the base image +FROM dhi.io/deno:2 + +# Set the working directory +WORKDIR /app + +# Copy server code into the container +COPY server.ts . + +# Set permissions (optional but recommended for security) +USER deno + +# Expose port 8000 +EXPOSE 8000 + +# Run the Deno server +CMD ["run", "--allow-net", "server.ts"] +``` + +{{< /tab >}} +{{< tab name="Using the official image" >}} + +Using the Docker Official Image is straightforward. In the following Dockerfile, you'll notice that the `FROM` instruction uses `denoland/deno:latest` as the base image. + +This is the official image for Deno. This image is [available on the Docker Hub](https://hub.docker.com/r/denoland/deno). + +```dockerfile +# Use the official Deno image +FROM denoland/deno:latest + +# Set the working directory +WORKDIR /app + +# Copy server code into the container +COPY server.ts . + +# Set permissions (optional but recommended for security) +USER deno + +# Expose port 8000 +EXPOSE 8000 + +# Run the Deno server +CMD ["run", "--allow-net", "server.ts"] +``` + +{{< /tab >}} +{{< /tabs >}} + +In addition to specifying the base image, the Dockerfile also: + +- Sets the working directory in the container to `/app`. +- Copies `server.ts` into the container. +- Sets the user to `deno` to run the application as a non-root user. +- Exposes port 8000 to allow traffic to the application. +- Runs the Deno server using the `CMD` instruction. +- Uses the `--allow-net` flag to allow network access to the application. The `server.ts` file uses the Oak framework to create a simple API that listens on port 8000. + +### Run the application + +Make sure you are in the `deno-docker` directory. Run the following command in a terminal to build and run the application. + +```console +$ docker compose up --build +``` + +Open a browser and view the application at [http://localhost:8000](http://localhost:8000). You will see a message `{"Status" : "OK"}` in the browser. + +In the terminal, press `ctrl`+`c` to stop the application. + +#### Run the application in the background + +You can run the application detached from the terminal by adding the `-d` +option. Inside the `deno-docker` directory, run the following command +in a terminal. + +```console +$ docker compose up --build -d +``` + +Open a browser and view the application at [http://localhost:8000](http://localhost:8000). + + +In the terminal, run the following command to stop the application. + +```console +$ docker compose down +``` + +### Summary + +In this section, you learned how you can containerize and run your Deno +application using Docker. + +Related information: + + - [Dockerfile reference](/reference/dockerfile.md) + - [.dockerignore file](/reference/dockerfile.md#dockerignore-file) + - [Docker Compose overview](/manuals/compose/_index.md) + - [Compose file reference](/reference/compose-file/_index.md) + - [Docker Hardened Images](/dhi/) + +### Next steps + +In the next section, you'll learn how you can develop your application using +containers. + +## Use containers for Deno development + +### Prerequisites + +Complete [Containerize a Deno application](containerize.md). + +### Overview + +In this section, you'll learn how to set up a development environment for your containerized application. This includes: + +- Configuring Compose to automatically update your running Compose services as you edit and save your code + +### Get the sample application + +Clone the sample application to use with this guide. Open a terminal, change directory to a directory that you want to work in, and run the following command to clone the repository: + +```console +$ git clone https://github.com/dockersamples/docker-deno.git && cd docker-deno +``` + +### Automatically update services + +Use Compose Watch to automatically update your running Compose services as you +edit and save your code. For more details about Compose Watch, see [Use Compose +Watch](/manuals/compose/how-tos/file-watch.md). + +Open your `compose.yml` file in an IDE or text editor and then add the Compose Watch instructions. The following example shows how to add Compose Watch to your `compose.yml` file. + +```yaml {hl_lines="9-12",linenos=true} +services: + server: + image: deno-server + build: + context: . + dockerfile: Dockerfile + ports: + - "8000:8000" + develop: + watch: + - action: rebuild + path: . +``` + +Run the following command to run your application with Compose Watch. + +```console +$ docker compose watch +``` + +Now, if you modify your `server.ts` you will see the changes in real time without re-building the image. + +To test it out, open the `server.ts` file in your favorite text editor and change the message from `{"Status" : "OK"}` to `{"Status" : "Updated"}`. Save the file and refresh your browser at `http://localhost:8000`. You should see the updated message. + +Press `ctrl+c` in the terminal to stop your application. + +### Summary + +In this section, you also learned how to use Compose Watch to automatically rebuild and run your container when you update your code. + +Related information: + - [Compose file reference](/reference/compose-file/) + - [Compose file watch](/manuals/compose/how-tos/file-watch.md) + - [Multi-stage builds](/manuals/build/building/multi-stage.md) + +### Next steps + +In the next section, you'll take a look at how to set up a CI/CD pipeline using GitHub Actions. + +## Configure CI/CD for your Deno application + +### Prerequisites + +Complete all the previous sections of this guide, starting with [Containerize a Deno application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. + +### Overview + +In this section, you'll learn how to set up and use GitHub Actions to build and test your Docker image as well as push it to Docker Hub. You will complete the following steps: + +1. Create a new repository on GitHub. +2. Define the GitHub Actions workflow. +3. Run the workflow. + +### Step one: Create the repository + +Create a GitHub repository, configure the Docker Hub credentials, and push your source code. + +1. [Create a new repository](https://github.com/new) on GitHub. + +2. Open the repository **Settings**, and go to **Secrets and variables** > + **Actions**. + +3. Create a new **Repository variable** named `DOCKER_USERNAME` and your Docker ID as a value. + +4. Create a new [Personal Access Token (PAT)](/manuals/security/access-tokens.md#create-an-access-token)for Docker Hub. You can name this token `docker-tutorial`. Make sure access permissions include Read and Write. + +5. Add the PAT as a **Repository secret** in your GitHub repository, with the name + `DOCKERHUB_TOKEN`. + +6. In your local repository on your machine, run the following command to change + the origin to the repository you just created. Make sure you change + `your-username` to your GitHub username and `your-repository` to the name of + the repository you created. + + ```console + $ git remote set-url origin https://github.com/your-username/your-repository.git + ``` + +7. Run the following commands to stage, commit, and push your local repository to GitHub. + + ```console + $ git add -A + $ git commit -m "my commit" + $ git push -u origin main + ``` + +### Step two: Set up the workflow + +Set up your GitHub Actions workflow for building and pushing the image +to Docker Hub. + +1. Go to your repository on GitHub and then select the **Actions** tab. + +2. Select **set up a workflow yourself**. + + This takes you to a page for creating a new GitHub actions workflow file in + your repository, under `.github/workflows/main.yml` by default. + +3. In the editor window, copy and paste the following YAML configuration and commit the changes. + + ```yaml + name: ci + + on: + push: + branches: + - main + + jobs: + build: + runs-on: ubuntu-latest + steps: + - + name: Login to Docker Hub + uses: docker/login-action@{{% param "login_action_version" %}} + with: + username: ${{ vars.DOCKER_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + - + name: Set up Docker Buildx + uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} + - + name: Build and push + uses: docker/build-push-action@{{% param "build_push_action_version" %}} + with: + platforms: linux/amd64,linux/arm64 + push: true + tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest + ``` + + For more information about the YAML syntax for `docker/build-push-action`, + refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). + +### Step three: Run the workflow + +Save the workflow file and run the job. + +1. Select **Commit changes...** and push the changes to the `main` branch. + + After pushing the commit, the workflow starts automatically. + +2. Go to the **Actions** tab. It displays the workflow. + + Selecting the workflow shows you the breakdown of all the steps. + +3. When the workflow is complete, go to your + [repositories on Docker Hub](https://hub.docker.com/repositories). + + If you see the new repository in that list, it means the GitHub Actions + successfully pushed the image to Docker Hub. + +### Summary + +In this section, you learned how to set up a GitHub Actions workflow for your Deno application. + +Related information: + - [Introduction to GitHub Actions](/manuals/build/ci/github-actions/_index.md) + - [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) + +### Next steps + +Next, learn how you can locally test and debug your workloads on Kubernetes before deploying. + +## Test your Deno deployment + +### Prerequisites + +- Complete all the previous sections of this guide, starting with [Containerize a Deno application](containerize.md). +- [Turn on Kubernetes](/manuals//desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. + +### Overview + +In this section, you'll learn how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. This allows you to test and debug your workloads on Kubernetes locally before deploying. + +### Create a Kubernetes YAML file + +In your `deno-docker` directory, create a file named +`docker-kubernetes.yml`. Open the file in an IDE or text editor and add +the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker +username and the name of the repository that you created in [Configure CI/CD for +your Deno application](configure-ci-cd.md). + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: docker-deno-demo + namespace: default +spec: + replicas: 1 + selector: + matchLabels: + app: deno-api + template: + metadata: + labels: + app: deno-api + spec: + containers: + - name: deno-api + image: DOCKER_USERNAME/REPO_NAME + imagePullPolicy: Always +--- +apiVersion: v1 +kind: Service +metadata: + name: service-entrypoint + namespace: default +spec: + type: NodePort + selector: + app: deno-api + ports: + - port: 8000 + targetPort: 8000 + nodePort: 30001 +``` + +In this Kubernetes YAML file, there are two objects, separated by the `---`: + + - A Deployment, describing a scalable group of identical pods. In this case, + you'll get just one replica, or copy of your pod. That pod, which is + described under `template`, has just one container in it. The + container is created from the image built by GitHub Actions in [Configure CI/CD for + your Deno application](configure-ci-cd.md). + - A NodePort service, which will route traffic from port 30001 on your host to + port 8000 inside the pods it routes to, allowing you to reach your app + from the network. + +To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). + +### Deploy and check your application + +1. In a terminal, navigate to `deno-docker` and deploy your application to + Kubernetes. + + ```console + $ kubectl apply -f docker-kubernetes.yml + ``` + + You should see output that looks like the following, indicating your Kubernetes objects were created successfully. + + ```text + deployment.apps/docker-deno-demo created + service/service-entrypoint created + ``` + +2. Make sure everything worked by listing your deployments. + + ```console + $ kubectl get deployments + ``` + + Your deployment should be listed as follows: + + ```shell + NAME READY UP-TO-DATE AVAILABLE AGE + docker-deno-demo 1/1 1 1 10s + ``` + + This indicates all one of the pods you asked for in your YAML are up and running. Do the same check for your services. + + ```console + $ kubectl get services + ``` + + You should get output like the following. + + ```shell + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + kubernetes ClusterIP 10.96.0.1 443/TCP 88m + service-entrypoint NodePort 10.105.145.223 8000:30001/TCP 83s + ``` + + In addition to the default `kubernetes` service, you can see your `service-entrypoint` service, accepting traffic on port 30001/TCP. + +3. In a browser, visit the following address. You should see the message `{"Status" : "OK"}`. + + ```console + http://localhost:30001/ + ``` + +4. Run the following command to tear down your application. + + ```console + $ kubectl delete -f docker-kubernetes.yml + ``` + +### Summary + +In this section, you learned how to use Docker Desktop to deploy your Deno application to a fully-featured Kubernetes environment on your development machine. + +Related information: + - [Kubernetes documentation](https://kubernetes.io/docs/home/) + - [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) + - [Swarm mode overview](/manuals/engine/swarm/_index.md) diff --git a/content/guides/deno/configure-ci-cd.md b/content/guides/deno/configure-ci-cd.md deleted file mode 100644 index 8d62f21be40a..000000000000 --- a/content/guides/deno/configure-ci-cd.md +++ /dev/null @@ -1,131 +0,0 @@ ---- -title: Configure CI/CD for your Deno application -linkTitle: Configure CI/CD -weight: 40 -keywords: ci/cd, github actions, deno, shiny -description: Learn how to configure CI/CD using GitHub Actions for your Deno application. -aliases: -- /language/deno/configure-ci-cd/ ---- - -## Prerequisites - -Complete all the previous sections of this guide, starting with [Containerize a Deno application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. - -## Overview - -In this section, you'll learn how to set up and use GitHub Actions to build and test your Docker image as well as push it to Docker Hub. You will complete the following steps: - -1. Create a new repository on GitHub. -2. Define the GitHub Actions workflow. -3. Run the workflow. - -## Step one: Create the repository - -Create a GitHub repository, configure the Docker Hub credentials, and push your source code. - -1. [Create a new repository](https://github.com/new) on GitHub. - -2. Open the repository **Settings**, and go to **Secrets and variables** > - **Actions**. - -3. Create a new **Repository variable** named `DOCKER_USERNAME` and your Docker ID as a value. - -4. Create a new [Personal Access Token (PAT)](/manuals/security/access-tokens.md#create-an-access-token)for Docker Hub. You can name this token `docker-tutorial`. Make sure access permissions include Read and Write. - -5. Add the PAT as a **Repository secret** in your GitHub repository, with the name - `DOCKERHUB_TOKEN`. - -6. In your local repository on your machine, run the following command to change - the origin to the repository you just created. Make sure you change - `your-username` to your GitHub username and `your-repository` to the name of - the repository you created. - - ```console - $ git remote set-url origin https://github.com/your-username/your-repository.git - ``` - -7. Run the following commands to stage, commit, and push your local repository to GitHub. - - ```console - $ git add -A - $ git commit -m "my commit" - $ git push -u origin main - ``` - -## Step two: Set up the workflow - -Set up your GitHub Actions workflow for building and pushing the image -to Docker Hub. - -1. Go to your repository on GitHub and then select the **Actions** tab. - -2. Select **set up a workflow yourself**. - - This takes you to a page for creating a new GitHub actions workflow file in - your repository, under `.github/workflows/main.yml` by default. - -3. In the editor window, copy and paste the following YAML configuration and commit the changes. - - ```yaml - name: ci - - on: - push: - branches: - - main - - jobs: - build: - runs-on: ubuntu-latest - steps: - - - name: Login to Docker Hub - uses: docker/login-action@{{% param "login_action_version" %}} - with: - username: ${{ vars.DOCKER_USERNAME }} - password: ${{ secrets.DOCKERHUB_TOKEN }} - - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} - - - name: Build and push - uses: docker/build-push-action@{{% param "build_push_action_version" %}} - with: - platforms: linux/amd64,linux/arm64 - push: true - tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest - ``` - - For more information about the YAML syntax for `docker/build-push-action`, - refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). - -## Step three: Run the workflow - -Save the workflow file and run the job. - -1. Select **Commit changes...** and push the changes to the `main` branch. - - After pushing the commit, the workflow starts automatically. - -2. Go to the **Actions** tab. It displays the workflow. - - Selecting the workflow shows you the breakdown of all the steps. - -3. When the workflow is complete, go to your - [repositories on Docker Hub](https://hub.docker.com/repositories). - - If you see the new repository in that list, it means the GitHub Actions - successfully pushed the image to Docker Hub. - -## Summary - -In this section, you learned how to set up a GitHub Actions workflow for your Deno application. - -Related information: - - [Introduction to GitHub Actions](/manuals/build/ci/github-actions/_index.md) - - [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) - -## Next steps - -Next, learn how you can locally test and debug your workloads on Kubernetes before deploying. diff --git a/content/guides/deno/containerize.md b/content/guides/deno/containerize.md deleted file mode 100644 index a738c6dfba83..000000000000 --- a/content/guides/deno/containerize.md +++ /dev/null @@ -1,197 +0,0 @@ ---- -title: Containerize a Deno application -linkTitle: Containerize your app -weight: 10 -keywords: deno, containerize, initialize -description: Learn how to containerize a Deno application. -aliases: - - /language/deno/containerize/ ---- - -## Prerequisites - -* You have a [Git client](https://git-scm.com/downloads). The examples in this section use a command-line based Git client, but you can use any client. - -## Overview - -For a long time, Node.js has been the go-to runtime for server-side JavaScript applications. However, recent years have introduced new alternative runtimes, including [Deno](https://deno.land/). Like Node.js, Deno is a JavaScript and TypeScript runtime, but it takes a fresh approach with modern security features, a built-in standard library, and native support for TypeScript. - -Why develop Deno applications with Docker? Having a choice of runtimes is exciting, but managing multiple runtimes and their dependencies consistently across environments can be tricky. This is where Docker proves invaluable. Using containers to create and destroy environments on demand simplifies runtime management and ensures consistency. Additionally, as Deno continues to grow and evolve, Docker helps establish a reliable and reproducible development environment, minimizing setup challenges and streamlining the workflow. - -## Get the sample application - -Clone the sample application to use with this guide. Open a terminal, change -directory to a directory that you want to work in, and run the following -command to clone the repository: - -```console -$ git clone https://github.com/dockersamples/docker-deno.git && cd docker-deno -``` - -You should now have the following contents in your `deno-docker` directory. - -```text -├── deno-docker/ -│ ├── compose.yml -│ ├── Dockerfile -│ ├── LICENSE -│ ├── server.ts -│ └── README.md -``` - -## Understand the sample application - -The sample application is a simple Deno application that uses the Oak framework to create a simple API that returns a JSON response. The application listens on port 8000 and returns a message `{"Status" : "OK"}` when you access the application in a browser. - -```typescript -// server.ts -import { Application, Router } from "https://deno.land/x/oak@v12.0.0/mod.ts"; - -const app = new Application(); -const router = new Router(); - -// Define a route that returns JSON -router.get("/", (context) => { - context.response.body = { Status: "OK" }; - context.response.type = "application/json"; -}); - -app.use(router.routes()); -app.use(router.allowedMethods()); - -console.log("Server running on http://localhost:8000"); -await app.listen({ port: 8000 }); -``` - -## Create a Dockerfile - -Before creating a Dockerfile, you need to choose a base image. You can either use the [Deno Docker Official Image](https://hub.docker.com/r/denoland/deno) or a Docker Hardened Image (DHI) from the [Hardened Image catalog](https://hub.docker.com/hardened-images/catalog). - -Choosing DHI offers the advantage of a production-ready image that is lightweight and secure. For more information, see [Docker Hardened Images](https://docs.docker.com/dhi/). - -{{< tabs >}} -{{< tab name="Using Docker Hardened Images" >}} - -Docker Hardened Images (DHIs) are available for Deno in the [Docker Hardened Images catalog](https://hub.docker.com/hardened-images/catalog/dhi/deno). You can pull DHIs directly from the `dhi.io` registry. - -1. Sign in to the DHI registry: - - ```console - $ docker login dhi.io - ``` - -2. Pull the Deno DHI as `dhi.io/deno:2`. The tag (`2`) in this example refers to the version to the latest 2.x version of Deno. - - ```console - $ docker pull dhi.io/deno:2 - ``` - -For other available versions, refer to the [catalog](https://hub.docker.com/hardened-images/catalog/dhi/deno). - -```dockerfile -# Use the DHI Deno image as the base image -FROM dhi.io/deno:2 - -# Set the working directory -WORKDIR /app - -# Copy server code into the container -COPY server.ts . - -# Set permissions (optional but recommended for security) -USER deno - -# Expose port 8000 -EXPOSE 8000 - -# Run the Deno server -CMD ["run", "--allow-net", "server.ts"] -``` - -{{< /tab >}} -{{< tab name="Using the official image" >}} - -Using the Docker Official Image is straightforward. In the following Dockerfile, you'll notice that the `FROM` instruction uses `denoland/deno:latest` as the base image. - -This is the official image for Deno. This image is [available on the Docker Hub](https://hub.docker.com/r/denoland/deno). - -```dockerfile -# Use the official Deno image -FROM denoland/deno:latest - -# Set the working directory -WORKDIR /app - -# Copy server code into the container -COPY server.ts . - -# Set permissions (optional but recommended for security) -USER deno - -# Expose port 8000 -EXPOSE 8000 - -# Run the Deno server -CMD ["run", "--allow-net", "server.ts"] -``` - -{{< /tab >}} -{{< /tabs >}} - -In addition to specifying the base image, the Dockerfile also: - -- Sets the working directory in the container to `/app`. -- Copies `server.ts` into the container. -- Sets the user to `deno` to run the application as a non-root user. -- Exposes port 8000 to allow traffic to the application. -- Runs the Deno server using the `CMD` instruction. -- Uses the `--allow-net` flag to allow network access to the application. The `server.ts` file uses the Oak framework to create a simple API that listens on port 8000. - -## Run the application - -Make sure you are in the `deno-docker` directory. Run the following command in a terminal to build and run the application. - -```console -$ docker compose up --build -``` - -Open a browser and view the application at [http://localhost:8000](http://localhost:8000). You will see a message `{"Status" : "OK"}` in the browser. - -In the terminal, press `ctrl`+`c` to stop the application. - -### Run the application in the background - -You can run the application detached from the terminal by adding the `-d` -option. Inside the `deno-docker` directory, run the following command -in a terminal. - -```console -$ docker compose up --build -d -``` - -Open a browser and view the application at [http://localhost:8000](http://localhost:8000). - - -In the terminal, run the following command to stop the application. - -```console -$ docker compose down -``` - -## Summary - -In this section, you learned how you can containerize and run your Deno -application using Docker. - -Related information: - - - [Dockerfile reference](/reference/dockerfile.md) - - [.dockerignore file](/reference/dockerfile.md#dockerignore-file) - - [Docker Compose overview](/manuals/compose/_index.md) - - [Compose file reference](/reference/compose-file/_index.md) - - [Docker Hardened Images](/dhi/) - -## Next steps - -In the next section, you'll learn how you can develop your application using -containers. diff --git a/content/guides/deno/deploy.md b/content/guides/deno/deploy.md deleted file mode 100644 index 7e00828ec9f1..000000000000 --- a/content/guides/deno/deploy.md +++ /dev/null @@ -1,141 +0,0 @@ ---- -title: Test your Deno deployment -linkTitle: Test your deployment -weight: 50 -keywords: deploy, kubernetes, deno -description: Learn how to develop locally using Kubernetes -aliases: -- /language/deno/deploy/ ---- - -## Prerequisites - -- Complete all the previous sections of this guide, starting with [Containerize a Deno application](containerize.md). -- [Turn on Kubernetes](/manuals//desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. - -## Overview - -In this section, you'll learn how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. This allows you to test and debug your workloads on Kubernetes locally before deploying. - -## Create a Kubernetes YAML file - -In your `deno-docker` directory, create a file named -`docker-kubernetes.yml`. Open the file in an IDE or text editor and add -the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker -username and the name of the repository that you created in [Configure CI/CD for -your Deno application](configure-ci-cd.md). - -```yaml -apiVersion: apps/v1 -kind: Deployment -metadata: - name: docker-deno-demo - namespace: default -spec: - replicas: 1 - selector: - matchLabels: - app: deno-api - template: - metadata: - labels: - app: deno-api - spec: - containers: - - name: deno-api - image: DOCKER_USERNAME/REPO_NAME - imagePullPolicy: Always ---- -apiVersion: v1 -kind: Service -metadata: - name: service-entrypoint - namespace: default -spec: - type: NodePort - selector: - app: deno-api - ports: - - port: 8000 - targetPort: 8000 - nodePort: 30001 -``` - -In this Kubernetes YAML file, there are two objects, separated by the `---`: - - - A Deployment, describing a scalable group of identical pods. In this case, - you'll get just one replica, or copy of your pod. That pod, which is - described under `template`, has just one container in it. The - container is created from the image built by GitHub Actions in [Configure CI/CD for - your Deno application](configure-ci-cd.md). - - A NodePort service, which will route traffic from port 30001 on your host to - port 8000 inside the pods it routes to, allowing you to reach your app - from the network. - -To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). - -## Deploy and check your application - -1. In a terminal, navigate to `deno-docker` and deploy your application to - Kubernetes. - - ```console - $ kubectl apply -f docker-kubernetes.yml - ``` - - You should see output that looks like the following, indicating your Kubernetes objects were created successfully. - - ```text - deployment.apps/docker-deno-demo created - service/service-entrypoint created - ``` - -2. Make sure everything worked by listing your deployments. - - ```console - $ kubectl get deployments - ``` - - Your deployment should be listed as follows: - - ```shell - NAME READY UP-TO-DATE AVAILABLE AGE - docker-deno-demo 1/1 1 1 10s - ``` - - This indicates all one of the pods you asked for in your YAML are up and running. Do the same check for your services. - - ```console - $ kubectl get services - ``` - - You should get output like the following. - - ```shell - NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE - kubernetes ClusterIP 10.96.0.1 443/TCP 88m - service-entrypoint NodePort 10.105.145.223 8000:30001/TCP 83s - ``` - - In addition to the default `kubernetes` service, you can see your `service-entrypoint` service, accepting traffic on port 30001/TCP. - -3. In a browser, visit the following address. You should see the message `{"Status" : "OK"}`. - - ```console - http://localhost:30001/ - ``` - -4. Run the following command to tear down your application. - - ```console - $ kubectl delete -f docker-kubernetes.yml - ``` - -## Summary - -In this section, you learned how to use Docker Desktop to deploy your Deno application to a fully-featured Kubernetes environment on your development machine. - -Related information: - - [Kubernetes documentation](https://kubernetes.io/docs/home/) - - [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) - - [Swarm mode overview](/manuals/engine/swarm/_index.md) diff --git a/content/guides/deno/develop.md b/content/guides/deno/develop.md deleted file mode 100644 index d717823b634d..000000000000 --- a/content/guides/deno/develop.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -title: Use containers for Deno development -linkTitle: Develop your app -weight: 20 -keywords: deno, local, development -description: Learn how to develop your Deno application locally. -aliases: -- /language/deno/develop/ ---- - -## Prerequisites - -Complete [Containerize a Deno application](containerize.md). - -## Overview - -In this section, you'll learn how to set up a development environment for your containerized application. This includes: - -- Configuring Compose to automatically update your running Compose services as you edit and save your code - -## Get the sample application - -Clone the sample application to use with this guide. Open a terminal, change directory to a directory that you want to work in, and run the following command to clone the repository: - -```console -$ git clone https://github.com/dockersamples/docker-deno.git && cd docker-deno -``` - -## Automatically update services - -Use Compose Watch to automatically update your running Compose services as you -edit and save your code. For more details about Compose Watch, see [Use Compose -Watch](/manuals/compose/how-tos/file-watch.md). - -Open your `compose.yml` file in an IDE or text editor and then add the Compose Watch instructions. The following example shows how to add Compose Watch to your `compose.yml` file. - -```yaml {hl_lines="9-12",linenos=true} -services: - server: - image: deno-server - build: - context: . - dockerfile: Dockerfile - ports: - - "8000:8000" - develop: - watch: - - action: rebuild - path: . -``` - -Run the following command to run your application with Compose Watch. - -```console -$ docker compose watch -``` - -Now, if you modify your `server.ts` you will see the changes in real time without re-building the image. - -To test it out, open the `server.ts` file in your favorite text editor and change the message from `{"Status" : "OK"}` to `{"Status" : "Updated"}`. Save the file and refresh your browser at `http://localhost:8000`. You should see the updated message. - -Press `ctrl+c` in the terminal to stop your application. - -## Summary - -In this section, you also learned how to use Compose Watch to automatically rebuild and run your container when you update your code. - -Related information: - - [Compose file reference](/reference/compose-file/) - - [Compose file watch](/manuals/compose/how-tos/file-watch.md) - - [Multi-stage builds](/manuals/build/building/multi-stage.md) - -## Next steps - -In the next section, you'll take a look at how to set up a CI/CD pipeline using GitHub Actions. diff --git a/content/guides/dex.md b/content/guides/dex.md index 0aaae8021195..b8c6b3a2f504 100644 --- a/content/guides/dex.md +++ b/content/guides/dex.md @@ -4,9 +4,8 @@ description: &desc Mocking OAuth services in testing with Dex keywords: Dex, container-supported development linktitle: Mocking OAuth services with Dex summary: *desc -tags: [app-dev, distributed-systems] -languages: [] params: + tags: [testing] time: 10 minutes --- diff --git a/content/guides/dhi-backstage.md b/content/guides/dhi-backstage.md index 01bf4f7f986a..ec1b7e06049f 100644 --- a/content/guides/dhi-backstage.md +++ b/content/guides/dhi-backstage.md @@ -3,8 +3,8 @@ title: Secure a Backstage application with Docker Hardened Images description: Secure a Backstage developer portal using Docker Hardened Images, covering native module compilation, Socket Firewall protection, and distroless runtime images. summary: Learn how to secure a Backstage developer portal using Docker Hardened Images (DHI), handle native module compilation with better-sqlite3, add Socket Firewall protection during dependency installation, and produce a distroless runtime image using DHI customizations. keywords: docker hardened images, dhi, backstage, CNCF, developer portal, node.js, native modules, sqlite, better-sqlite3, distroless, socket firewall, dhictl, multi-stage build -tags: ["Docker Hardened Images", "dhi"] params: + tags: [security] proficiencyLevel: Intermediate time: 45 minutes prerequisites: diff --git a/content/guides/dhi-from-doi.md b/content/guides/dhi-from-doi.md deleted file mode 100644 index d0b367753d0e..000000000000 --- a/content/guides/dhi-from-doi.md +++ /dev/null @@ -1,13 +0,0 @@ ---- -title: Migrate to DHI from Docker Official Images -summary: Step-by-step guide to migrate from Docker Official Images to Docker Hardened Images -keywords: docker hardened images, dhi, docker official images, migration, secure images -type: redirect -target: /dhi/migration/migrate-from-doi/ -tags: [dhi] -languages: [] -params: - time: 10 minutes - featured: true - image: /images/guides/dhi-migrate-doi.webp ---- diff --git a/content/guides/dhi-from-wolfi.md b/content/guides/dhi-from-wolfi.md deleted file mode 100644 index 166ed977de86..000000000000 --- a/content/guides/dhi-from-wolfi.md +++ /dev/null @@ -1,13 +0,0 @@ ---- -title: Migrate to DHI from Wolfi -summary: Step-by-step guide to migrate from Wolfi to Docker Hardened Images -keywords: docker hardened images, dhi, wolfi, migration, secure images, distroless -type: redirect -target: /dhi/migration/migrate-from-wolfi/ -tags: [dhi] -languages: [] -params: - time: 10 minutes - featured: true - image: /images/guides/dhi-migrate-wolfi.webp ---- diff --git a/content/guides/dhi-go-example.md b/content/guides/dhi-go-example.md deleted file mode 100644 index f3adf7d67b51..000000000000 --- a/content/guides/dhi-go-example.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -title: Migrate a Go app to DHI -summary: | - Example showing how to migrate a Go application to Docker Hardened Images -keywords: docker hardened images, dhi, go, golang, migration, secure images -type: redirect -target: /dhi/migration/examples/go/ -tags: [dhi] -languages: [] -params: - time: 10 minutes - featured: true - image: /images/guides/dhi-examples-go.webp ---- diff --git a/content/guides/dhi-nodejs-example.md b/content/guides/dhi-nodejs-example.md deleted file mode 100644 index da22a6ebed23..000000000000 --- a/content/guides/dhi-nodejs-example.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -title: Migrate a Node.js app to DHI -summary: | - Example showing how to migrate a Node.js application to Docker Hardened - Images -keywords: docker hardened images, dhi, node.js, nodejs, migration, secure images -type: redirect -target: /dhi/migration/examples/node/ -tags: [dhi] -languages: [] -params: - time: 10 minutes - featured: true - image: /images/guides/dhi-examples-nodejs.webp ---- diff --git a/content/guides/dhi-openshift.md b/content/guides/dhi-openshift.md index d347f604178b..57d9295cb017 100644 --- a/content/guides/dhi-openshift.md +++ b/content/guides/dhi-openshift.md @@ -3,16 +3,15 @@ title: Use Docker Hardened Images with Red Hat OpenShift description: Deploy Docker Hardened Images on Red Hat OpenShift Container Platform, covering Security Context Constraints, arbitrary user ID assignment, file permissions, and best practices. summary: Learn how to deploy Docker Hardened Images (DHI) on Red Hat OpenShift, configure Security Context Constraints, handle arbitrary user ID assignment, and set file permissions for both runtime and development image variants. keywords: docker hardened images, dhi, openshift, OCP, SCC, security context constraints, non-root, distroless, containers, red hat -tags: ["Docker Hardened Images", "dhi"] params: + tags: [security] proficiencyLevel: Intermediate time: 30 minutes prerequisites: - An OpenShift cluster (version 4.11 or later recommended) - The oc CLI authenticated to your cluster - A Docker Hub account with access to Docker Hardened Images - - Familiarity with OpenShift Security Context Constraints (SCCs) ---- + - Familiarity with OpenShift Security Context Constraints (SCCs)--- Docker Hardened Images (DHI) can be deployed on Red Hat OpenShift Container Platform, but OpenShift’s security model differs from standard Kubernetes in diff --git a/content/guides/dhi-python-example.md b/content/guides/dhi-python-example.md deleted file mode 100644 index e45b5cd9a0b4..000000000000 --- a/content/guides/dhi-python-example.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -title: Migrate a Python app to DHI -summary: | - Example showing how to migrate a Python application to Docker Hardened Images -keywords: docker hardened images, dhi, python, migration, secure images -type: redirect -target: /dhi/migration/examples/python/ -tags: [dhi] -languages: [] -params: - time: 10 minutes - featured: true - image: /images/guides/dhi-examples-python.webp ---- diff --git a/content/guides/dhi-vex-walkthrough.md b/content/guides/dhi-vex-walkthrough.md index 9ffbde76f822..ae17d4815b4a 100644 --- a/content/guides/dhi-vex-walkthrough.md +++ b/content/guides/dhi-vex-walkthrough.md @@ -7,8 +7,8 @@ summary: > Scan a Docker Hardened Image with and without VEX and audit every suppression and its justification. keywords: vex, openvex, not_affected, under_investigation, affected, cve, docker scout, dhi, vulnerability -tags: [dhi] params: + tags: [security] proficiencyLevel: Intermediate time: 25 minutes --- diff --git a/content/guides/django.md b/content/guides/django.md index fa242ece8417..837de854dc68 100644 --- a/content/guides/django.md +++ b/content/guides/django.md @@ -8,9 +8,8 @@ summary: | You'll scaffold the project with uv, create a production-ready Dockerfile using a Docker Hardened Image, then add a development stage and Compose Watch for fast iteration. -languages: [python] -tags: [dhi] params: + tags: [security] time: 25 minutes --- diff --git a/content/guides/docker-build-cloud/_index.md b/content/guides/docker-build-cloud/_index.md index 2c7eca2cf459..211f5fbfe8c7 100644 --- a/content/guides/docker-build-cloud/_index.md +++ b/content/guides/docker-build-cloud/_index.md @@ -8,23 +8,19 @@ summary: | Build applications up to 39x faster using cloud-based resources, shared team cache, and native multi-architecture support. keywords: docker build cloud, cloud builds, multi-architecture, shared cache, ci/cd, build performance -tags: [product-demo] aliases: - /learning-paths/docker-build-cloud/ + - /guides/docker-build-cloud/ci/ + - /guides/docker-build-cloud/common-questions/ + - /guides/docker-build-cloud/dev/ + - /guides/docker-build-cloud/why/ params: + tags: [cicd] image: images/learning-paths/build-cloud.png time: 10 minutes - resource_links: - - title: Product page - url: https://www.docker.com/products/build-cloud/ - - title: Docker Build Cloud overview - url: /build-cloud/ - - title: Subscriptions and features - url: "https://www.docker.com/pricing?ref=Docs&refAction=DocsGuidesBuildCloud" - - title: Using Docker Build Cloud - url: /build-cloud/usage/ --- + 98% of developers spend up to an hour every day waiting for builds to finish @@ -58,3 +54,120 @@ productivity, reduce frustrations, and help you shorten the release cycle. Works well with Docker Compose, GitHub Actions, and other CI solutions
+ +## Why Docker Build Cloud? + +Docker Build Cloud is a service that lets you build container images faster, +both locally and in CI. Builds run on cloud infrastructure optimally +dimensioned for your workloads, with no configuration required. The service +uses a remote build cache, ensuring fast builds anywhere and for all team +members. + +Docker Build Cloud provides several benefits over local builds: + +- Improved build speed +- Shared build cache +- Native multi-platform builds + +There’s no need to worry about managing builders or infrastructure — simply +connect to your builders and start building. Each cloud builder provisioned to +an organization is completely isolated to a single Amazon EC2 instance, with a +dedicated EBS volume for build cache and encryption in transit. That means +there are no shared processes or data between cloud builders. + +{{< youtube-embed "8AqKhEO2PQA" >}} + +
+ +## Demo: set up and use Docker Build Cloud in development + +With Docker Build Cloud, you can easily shift the build workload from local machines +to the cloud, helping you achieve faster build times, especially for multi-platform builds. + +In this demo, you'll see: + +- How to setup the builder locally +- How to use Docker Build Cloud with Docker Compose +- How the image cache speeds up builds for others on your team + +{{< youtube-embed "oPGq2AP5OtQ" >}} + +
+ +## Demo: Using Docker Build Cloud in CI + +Docker Build Cloud can significantly decrease the time it takes for your CI builds +take to run, saving you time and money. + +Since the builds run remotely, your CI runner can still use the Docker tooling CLI +without needing elevated permissions, making your builds more secure by default. + +In this demo, you will see: + +- How to integrate Docker Build Cloud into a variety of CI platforms +- How to use Docker Build Cloud in GitHub Actions to build multi-architecture images +- Speed differences between a workflow using Docker Build Cloud and a workflow running natively +- How to use Docker Build Cloud in a GitLab Pipeline + +{{< youtube-embed "wvLdInoVBGg" >}} + +
+ +## Common challenges and questions + +#### Is Docker Build Cloud a standalone product or a part of Docker Desktop? + +Docker Build Cloud is a service that can be used both with Docker Desktop and +standalone. It lets you build your container images faster, both locally and in +CI, with builds running on cloud infrastructure. The service uses a remote +build cache, ensuring fast builds anywhere and for all team members. + +When used with Docker Desktop, the [Builds view](/desktop/use-desktop/builds/) +works with Docker Build Cloud out-of-the-box. It shows information about your +builds and those initiated by your team members using the same builder, +enabling collaborative troubleshooting. + +To use Docker Build Cloud without Docker Desktop, you must +[download and install](/build-cloud/setup/#use-docker-build-cloud-without-docker-desktop) +a version of Buildx with support for Docker Build Cloud (the `cloud` driver). +If you plan on building with Docker Build Cloud using the `docker compose +build` command, you also need a version of Docker Compose that supports Docker +Build Cloud. + +#### How does Docker Build Cloud work with Docker Compose? + +Docker Compose works out of the box with Docker Build Cloud. Install the Docker +Build Cloud-compatible client (buildx) and it works with both commands. + +#### How many minutes are included in Docker Build Cloud Team plans? + +Pricing details for Docker Build Cloud can be found on the [pricing page](https://www.docker.com/pricing?ref=Docs&refAction=DocsGuidesBuildCloudFaq). + +#### I’m a Docker personal user. Can I try Docker Build Cloud? + +Docker subscribers (Pro, Team, Business) receive a set number of minutes each +month, shared across the account, to use Build Cloud. + +If you do not have a Docker subscription, you may sign up for a free Personal +account and start a trial of Docker Build Cloud. Personal accounts are limited to a +single user. + +For teams to receive the shared cache benefit, they must either be on a Docker +Team or Docker Business subscription. + +#### Does Docker Build Cloud support CI platforms? Does it work with GitHub Actions? + +Yes, Docker Build Cloud can be used with various CI platforms including GitHub +Actions, CircleCI, Jenkins, and others. It can speed up your build pipelines, +which means less time spent waiting and context switching. + +Docker Build Cloud can be used with GitHub Actions to automate your build, +test, and deployment pipeline. Docker provides a set of official GitHub Actions +that you can use in your workflows. + +Using GitHub Actions with Docker Build Cloud is straightforward. With a +one-line change in your GitHub Actions configuration, everything else stays the +same. You don't need to create new pipelines. Learn more in the [CI +documentation](/build-cloud/ci/) for Docker Build Cloud. + +
diff --git a/content/guides/docker-build-cloud/ci.md b/content/guides/docker-build-cloud/ci.md deleted file mode 100644 index a13adb91004c..000000000000 --- a/content/guides/docker-build-cloud/ci.md +++ /dev/null @@ -1,23 +0,0 @@ ---- -title: "Demo: Using Docker Build Cloud in CI" -description: Learn how to use Docker Build Cloud to build your app faster in CI. -keywords: docker build cloud, ci/cd, github actions, cloud builds, build performance -weight: 30 ---- - -Docker Build Cloud can significantly decrease the time it takes for your CI builds -take to run, saving you time and money. - -Since the builds run remotely, your CI runner can still use the Docker tooling CLI -without needing elevated permissions, making your builds more secure by default. - -In this demo, you will see: - -- How to integrate Docker Build Cloud into a variety of CI platforms -- How to use Docker Build Cloud in GitHub Actions to build multi-architecture images -- Speed differences between a workflow using Docker Build Cloud and a workflow running natively -- How to use Docker Build Cloud in a GitLab Pipeline - -{{< youtube-embed "wvLdInoVBGg" >}} - -
diff --git a/content/guides/docker-build-cloud/common-questions.md b/content/guides/docker-build-cloud/common-questions.md deleted file mode 100644 index eb340b4cb2fd..000000000000 --- a/content/guides/docker-build-cloud/common-questions.md +++ /dev/null @@ -1,63 +0,0 @@ ---- -title: Common challenges and questions -description: Explore common challenges and questions related to Docker Build Cloud. -keywords: docker build cloud, faq, troubleshooting, cloud builds, pricing -weight: 40 ---- - -### Is Docker Build Cloud a standalone product or a part of Docker Desktop? - -Docker Build Cloud is a service that can be used both with Docker Desktop and -standalone. It lets you build your container images faster, both locally and in -CI, with builds running on cloud infrastructure. The service uses a remote -build cache, ensuring fast builds anywhere and for all team members. - -When used with Docker Desktop, the [Builds view](/desktop/use-desktop/builds/) -works with Docker Build Cloud out-of-the-box. It shows information about your -builds and those initiated by your team members using the same builder, -enabling collaborative troubleshooting. - -To use Docker Build Cloud without Docker Desktop, you must -[download and install](/build-cloud/setup/#use-docker-build-cloud-without-docker-desktop) -a version of Buildx with support for Docker Build Cloud (the `cloud` driver). -If you plan on building with Docker Build Cloud using the `docker compose -build` command, you also need a version of Docker Compose that supports Docker -Build Cloud. - -### How does Docker Build Cloud work with Docker Compose? - -Docker Compose works out of the box with Docker Build Cloud. Install the Docker -Build Cloud-compatible client (buildx) and it works with both commands. - -### How many minutes are included in Docker Build Cloud Team plans? - -Pricing details for Docker Build Cloud can be found on the [pricing page](https://www.docker.com/pricing?ref=Docs&refAction=DocsGuidesBuildCloudFaq). - -### I’m a Docker personal user. Can I try Docker Build Cloud? - -Docker subscribers (Pro, Team, Business) receive a set number of minutes each -month, shared across the account, to use Build Cloud. - -If you do not have a Docker subscription, you may sign up for a free Personal -account and start a trial of Docker Build Cloud. Personal accounts are limited to a -single user. - -For teams to receive the shared cache benefit, they must either be on a Docker -Team or Docker Business subscription. - -### Does Docker Build Cloud support CI platforms? Does it work with GitHub Actions? - -Yes, Docker Build Cloud can be used with various CI platforms including GitHub -Actions, CircleCI, Jenkins, and others. It can speed up your build pipelines, -which means less time spent waiting and context switching. - -Docker Build Cloud can be used with GitHub Actions to automate your build, -test, and deployment pipeline. Docker provides a set of official GitHub Actions -that you can use in your workflows. - -Using GitHub Actions with Docker Build Cloud is straightforward. With a -one-line change in your GitHub Actions configuration, everything else stays the -same. You don't need to create new pipelines. Learn more in the [CI -documentation](/build-cloud/ci/) for Docker Build Cloud. - -
diff --git a/content/guides/docker-build-cloud/dev.md b/content/guides/docker-build-cloud/dev.md deleted file mode 100644 index 33ba82ec62b9..000000000000 --- a/content/guides/docker-build-cloud/dev.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -title: "Demo: set up and use Docker Build Cloud in development" -description: Learn how to use Docker Build Cloud for local builds. -keywords: docker build cloud, local development, cloud builds, multi-platform, build performance -weight: 20 ---- - -With Docker Build Cloud, you can easily shift the build workload from local machines -to the cloud, helping you achieve faster build times, especially for multi-platform builds. - -In this demo, you'll see: - -- How to setup the builder locally -- How to use Docker Build Cloud with Docker Compose -- How the image cache speeds up builds for others on your team - -{{< youtube-embed "oPGq2AP5OtQ" >}} - -
diff --git a/content/guides/docker-build-cloud/why.md b/content/guides/docker-build-cloud/why.md deleted file mode 100644 index cc1e5599d7ca..000000000000 --- a/content/guides/docker-build-cloud/why.md +++ /dev/null @@ -1,28 +0,0 @@ ---- -title: Why Docker Build Cloud? -description: Learn how Docker Build Cloud makes your builds faster. -keywords: docker build cloud, cloud builds, shared cache, multi-architecture, build performance -weight: 10 ---- - -Docker Build Cloud is a service that lets you build container images faster, -both locally and in CI. Builds run on cloud infrastructure optimally -dimensioned for your workloads, with no configuration required. The service -uses a remote build cache, ensuring fast builds anywhere and for all team -members. - -Docker Build Cloud provides several benefits over local builds: - -- Improved build speed -- Shared build cache -- Native multi-platform builds - -There’s no need to worry about managing builders or infrastructure — simply -connect to your builders and start building. Each cloud builder provisioned to -an organization is completely isolated to a single Amazon EC2 instance, with a -dedicated EBS volume for build cache and encryption in transit. That means -there are no shared processes or data between cloud builders. - -{{< youtube-embed "8AqKhEO2PQA" >}} - -
diff --git a/content/guides/docker-compose/_index.md b/content/guides/docker-compose/_index.md index 4b715d87b845..c0d330db034e 100644 --- a/content/guides/docker-compose/_index.md +++ b/content/guides/docker-compose/_index.md @@ -6,27 +6,18 @@ summary: | Docker applications. description: Learn how to use Docker Compose to define and run multi-container Docker applications. keywords: docker compose, multi-container, compose file, services, orchestration, yaml -tags: [product-demo] aliases: - /learning-paths/docker-compose/ + - /guides/docker-compose/common-questions/ + - /guides/docker-compose/setup/ + - /guides/docker-compose/why/ params: + tags: [cicd] image: images/learning-paths/compose.png time: 10 minutes - resource_links: - - title: Overview of Docker Compose CLI - url: /reference/cli/docker/compose/ - - title: Overview of Docker Compose - url: /compose/ - - title: How Compose works - url: /compose/intro/compose-application-model/ - - title: Using profiles with Compose - url: /compose/how-tos/profiles/ - - title: Control startup and shutdown order with Compose - url: /compose/how-tos/startup-order/ - - title: Compose Build Specification - url: /compose/compose-file/build/ --- + Developers face challenges with multi-container Docker applications, including complex configuration, dependency management, and maintaining consistent environments. Networking, resource allocation, data persistence, logging, and @@ -61,3 +52,109 @@ orchestrated. Works well with Docker CLI, CI/CD tools, and container orchestration tools.
+ +## Why Docker Compose? + +Docker Compose is an essential tool for defining and running multi-container +Docker applications. Docker Compose simplifies the Docker experience, making it +easier for developers to create, manage, and deploy applications by using YAML +files to configure application services. + +Docker Compose provides several benefits: + +- Lets you define multi-container applications in a single YAML file. +- Ensures consistent environments across development, testing, and production. +- Manages the startup and linking of multiple containers effortlessly. +- Streamlines development workflows and reduces setup time. +- Ensures that each service runs in its own container, avoiding conflicts. + +{{< youtube-embed 2EqarOM2V4U >}} + +
+ +## Demo: set up and use Docker Compose + +This Docker Compose demo shows how to orchestrate a multi-container application +environment, streamlining development and deployment processes. + +- Compare Docker Compose to the `docker run` command +- Configure a multi-container web app using a Compose file +- Run a multi-container web app using one command + +{{< youtube-embed P5RBKmOLPH4 >}} + +
+ +## Common challenges and questions + + + +#### Do I need to maintain a separate Compose file for my development, testing, and staging environments? + +You don't necessarily need to maintain entirely separate Compose files for your +development, testing, and staging environments. You can define all your +services in a single Compose file (`compose.yaml`). You can use profiles to +group service configurations specific to each environment (`dev`, `test`, +`staging`). + +When you need to spin up an environment, you can activate the corresponding +profiles. For example, to set up the development environment: + +```console +$ docker compose --profile dev up +``` + +This command starts only the services associated with the `dev` profile, +leaving the rest inactive. + +For more information on using profiles, see [Using profiles with +Compose](/compose/how-tos/profiles/). + +#### How can I enforce the database service to start up before the frontend service? + +Docker Compose ensures services start in a specific order by using the +`depends_on` property. This tells Compose to start the database service before +even attempting to launch the frontend service. This is crucial since +applications often rely on databases being ready for connections. + +However, `depends_on` only guarantees the order, not that the database is fully +initialized. For a more robust approach, especially if your application relies +on a prepared database (e.g., after migrations), consider [health +checks](/reference/compose-file/services.md#healthcheck). Here, you can +configure the frontend to wait until the database passes its health check +before starting. This ensures the database is not only up but also ready to +handle requests. + +For more information on setting the startup order of your services, see +[Control startup and shutdown order in Compose](/compose/how-tos/startup-order/). + +#### Can I use Compose to build a Docker image? + +Yes, you can use Docker Compose to build Docker images. Docker Compose is a +tool for defining and running multi-container applications. Even if your +application isn't a multi-container application, Docker Compose can make it +easier to run by defining all the `docker run` options in a file. + +To use Compose, you need a `compose.yaml` file. In this file, you can specify +the build context and Dockerfile for each service. When you run the command +`docker compose up --build`, Docker Compose will build the images for each +service and then start the containers. + +For more information on building Docker images using Compose, see the [Compose +Build Specification](/compose/compose-file/build/). + +#### What is the difference between Docker Compose and Dockerfile? + +A Dockerfile provides instructions to build a container image while a Compose +file defines your running containers. Quite often, a Compose file references a +Dockerfile to build an image to use for a particular service. + +#### What is the difference between the `docker compose up` and `docker compose run` commands? + +The `docker compose up` command creates and starts all your services. It's +perfect for launching your development environment or running the entire +application. The `docker compose run` command focuses on individual services. +It starts a specified service along with its dependencies, allowing you to run +tests or perform one-off tasks within that container. + +
diff --git a/content/guides/docker-compose/common-questions.md b/content/guides/docker-compose/common-questions.md deleted file mode 100644 index ba8f00154948..000000000000 --- a/content/guides/docker-compose/common-questions.md +++ /dev/null @@ -1,78 +0,0 @@ ---- -title: Common challenges and questions -description: Explore common challenges and questions related to Docker Compose. -keywords: docker compose, faq, troubleshooting, environments, multi-container -weight: 30 ---- - - - -### Do I need to maintain a separate Compose file for my development, testing, and staging environments? - -You don't necessarily need to maintain entirely separate Compose files for your -development, testing, and staging environments. You can define all your -services in a single Compose file (`compose.yaml`). You can use profiles to -group service configurations specific to each environment (`dev`, `test`, -`staging`). - -When you need to spin up an environment, you can activate the corresponding -profiles. For example, to set up the development environment: - -```console -$ docker compose --profile dev up -``` - -This command starts only the services associated with the `dev` profile, -leaving the rest inactive. - -For more information on using profiles, see [Using profiles with -Compose](/compose/how-tos/profiles/). - -### How can I enforce the database service to start up before the frontend service? - -Docker Compose ensures services start in a specific order by using the -`depends_on` property. This tells Compose to start the database service before -even attempting to launch the frontend service. This is crucial since -applications often rely on databases being ready for connections. - -However, `depends_on` only guarantees the order, not that the database is fully -initialized. For a more robust approach, especially if your application relies -on a prepared database (e.g., after migrations), consider [health -checks](/reference/compose-file/services.md#healthcheck). Here, you can -configure the frontend to wait until the database passes its health check -before starting. This ensures the database is not only up but also ready to -handle requests. - -For more information on setting the startup order of your services, see -[Control startup and shutdown order in Compose](/compose/how-tos/startup-order/). - -### Can I use Compose to build a Docker image? - -Yes, you can use Docker Compose to build Docker images. Docker Compose is a -tool for defining and running multi-container applications. Even if your -application isn't a multi-container application, Docker Compose can make it -easier to run by defining all the `docker run` options in a file. - -To use Compose, you need a `compose.yaml` file. In this file, you can specify -the build context and Dockerfile for each service. When you run the command -`docker compose up --build`, Docker Compose will build the images for each -service and then start the containers. - -For more information on building Docker images using Compose, see the [Compose -Build Specification](/compose/compose-file/build/). - -### What is the difference between Docker Compose and Dockerfile? - -A Dockerfile provides instructions to build a container image while a Compose -file defines your running containers. Quite often, a Compose file references a -Dockerfile to build an image to use for a particular service. - -### What is the difference between the `docker compose up` and `docker compose run` commands? - -The `docker compose up` command creates and starts all your services. It's -perfect for launching your development environment or running the entire -application. The `docker compose run` command focuses on individual services. -It starts a specified service along with its dependencies, allowing you to run -tests or perform one-off tasks within that container. - -
diff --git a/content/guides/docker-compose/setup.md b/content/guides/docker-compose/setup.md deleted file mode 100644 index 15d168b0d0dd..000000000000 --- a/content/guides/docker-compose/setup.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -title: "Demo: set up and use Docker Compose" -description: Learn how to get started with Docker Compose. -keywords: docker compose, getting started, multi-container, demo, orchestration -weight: 20 ---- - -This Docker Compose demo shows how to orchestrate a multi-container application -environment, streamlining development and deployment processes. - -- Compare Docker Compose to the `docker run` command -- Configure a multi-container web app using a Compose file -- Run a multi-container web app using one command - -{{< youtube-embed P5RBKmOLPH4 >}} - -
diff --git a/content/guides/docker-compose/why.md b/content/guides/docker-compose/why.md deleted file mode 100644 index f3ef90668a84..000000000000 --- a/content/guides/docker-compose/why.md +++ /dev/null @@ -1,23 +0,0 @@ ---- -title: Why Docker Compose? -description: Learn how Docker Compose can help you simplify app development. -keywords: docker compose, multi-container, yaml, services, orchestration, application -weight: 10 ---- - -Docker Compose is an essential tool for defining and running multi-container -Docker applications. Docker Compose simplifies the Docker experience, making it -easier for developers to create, manage, and deploy applications by using YAML -files to configure application services. - -Docker Compose provides several benefits: - -- Lets you define multi-container applications in a single YAML file. -- Ensures consistent environments across development, testing, and production. -- Manages the startup and linking of multiple containers effortlessly. -- Streamlines development workflows and reduces setup time. -- Ensures that each service runs in its own container, avoiding conflicts. - -{{< youtube-embed 2EqarOM2V4U >}} - -
diff --git a/content/guides/docker-scout/_index.md b/content/guides/docker-scout/_index.md index a8bbb683ebeb..f9ad6acb9978 100644 --- a/content/guides/docker-scout/_index.md +++ b/content/guides/docker-scout/_index.md @@ -9,21 +9,24 @@ description: | vulnerability detection and remediation, ensuring compliance, and protecting your development workflow. keywords: docker scout, container security, vulnerability scanning, sbom, supply chain, remediation -tags: [product-demo] aliases: - /learning-paths/docker-scout/ + - /scout/concepts/s3c/ + - /scout/concepts/sbom/ + - /guides/docker-scout/attestations/ + - /guides/docker-scout/common-questions/ + - /guides/docker-scout/demo/ + - /guides/docker-scout/remediation/ + - /guides/docker-scout/s3c/ + - /guides/docker-scout/sbom/ + - /guides/docker-scout/why/ params: + tags: [security] image: images/learning-paths/scout.png time: 20 minutes - resource_links: - - title: Docker Scout overview - url: /scout/ - - title: Docker Scout quickstart - url: /scout/quickstart/ - - title: Install Docker Scout - url: /scout/install/ --- + When container images are insecure, significant risks can arise. Around 60% of organizations have reported experiencing at least one security breach or vulnerability incident within a year, [resulting in operational @@ -63,3 +66,260 @@ other CI solutions. applications.
+ +## Why Docker Scout? + +{{< youtube-embed "-omsQ7Uqyc4" >}} + +Organizations face significant challenges from data breaches, +including financial losses, operational disruptions, and long-term damage to +brand reputation and customer trust. Docker Scout addresses critical problems +such as identifying insecure container images, preventing security breaches, +and reducing the risk of operational downtime due to vulnerabilities. + +Docker Scout provides several benefits: + +- Secure and trusted content +- A system of record for your Software Development Lifecycle (SDLC) +- Continuous security posture improvement + +Docker Scout offers automated vulnerability detection and remediation, helping +organizations identify and fix security issues in container images early in the +development process. It also integrates with popular development tools like +Docker Desktop and GitHub Actions, providing seamless security management and +compliance checks within existing workflows. + +
+ +## Docker Scout demo + +{{< youtube-embed "TkLwJ0p46W8" >}} + +Docker Scout has powerful features for enhancing containerized application +security and ensuring a robust software supply chain. + +- Define vulnerability remediation +- Discuss why remediation is essential to maintain the security and integrity + of containerized applications +- Discuss common vulnerabilities +- Implement remediation techniques: updating base images, applying patches, + removing unnecessary packages +- Verify and validate remediation efforts using Docker Scout + +
+ +## Software supply chain security + +{{< youtube-embed YzNK6E7APv0 >}} + +The term "software supply chain" refers to the end-to-end process of developing +and delivering software, from the development to deployment and maintenance. +Software supply chain security, or "S3C" for short, is the practice for +protecting the components and processes of the supply chain. + +S3C is a fundamental change in how organizations approach software security. +Traditionally in the software industry, security and compliance has been mostly +an afterthought, left to the software delivery or release phase. With S3C, +security is integrated into the entire software development lifecycle, from the +inner loop of development and testing, to the outer loop of shipping and +monitoring. + +Following industry best practices for software supply chain conduct is +important because it helps organizations protect their software from security +threats, compliance risks, and other vulnerabilities. Implementing a software +supply chain security framework improves visibility, collaboration, and +traceability of a project across stakeholders. This helps organizations detect, +respond to, and remediate threats more effectively. + +### Securing the software supply chain + +Building a secure software supply chain involves several key steps, such as: + +- Identify the software components and dependencies you use to build and run + your applications. +- Automate security testing throughout the software development lifecycle. +- Monitor your software supply chain for security threats. +- Implement security policies that govern how software is built, and the + components it contains. + +Managing the software supply chain is a complex task, especially in the modern +day where software is built using multiple components from different sources. +Organizations need to have a clear understanding of the software components +they use, and the security risks associated with them. + +### How Docker Scout is different + +Docker Scout is a platform designed to help organizations secure their software +supply chain. It provides tools and services for identifying and managing +software assets and policies, and automated remediation of security threats. + +Unlike traditional security tools that focus on scheduled, point-in-time scans +at specific stages in the software development lifecycle, Docker Scout uses a +modern event-driven model that spans the entire software supply chain. This +means that when a new vulnerability affecting your images is disclosed, your +updated risk assessment is available within seconds, and earlier in the +development process. + +Docker Scout works by analyzing the composition of your images to create a +Software Bill of Materials (SBOM). The SBOM is cross-referenced against the +security advisories to identify CVEs that affect your images. Docker Scout +integrates with [over 20 different security +advisories](/manuals/scout/deep-dive/advisory-db-sources.md), and updates its +vulnerability database in real-time. This ensures that your security posture is +represented using the latest available information. + +
+ +## Software Bill of Materials + +{{< youtube-embed PbS4y7C7h4A >}} + +A Bill of Materials (BOM) is a list of materials, parts, and the quantities of +each needed to manufacture a product. For example, a BOM for a computer might +list the motherboard, CPU, RAM, power supply, storage devices, case, and other +components, along with the quantities of each that are needed to build the +computer. + +A Software Bill of Materials (SBOM) is a list of all the components that make +up a piece of software. This includes open source and third-party components, +as well as any custom code that has been written for the software. An SBOM is +similar to a BOM for a physical product, but for software. + +In the context of software supply chain security, SBOMs can help with +identifying and mitigating security and compliance risks in software. By +knowing exactly what components are used in a piece of software, you can +quickly identify and patch vulnerabilities in your components, or determine if +a component is licensed in a way that is incompatible with your project. + +### Contents of an SBOM + +An SBOM typically includes the following information: + +- The name of the software, such as the name of a library or framework, that + the SBOM describes. +- The version of the software. +- The license under which the software is distributed. +- A list of other components that the software depends on. + +### How Docker Scout uses SBOMs + +Docker Scout uses SBOMs to determine the components that are used in a Docker +image. When you analyze an image, Docker Scout will either use the SBOM that is +attached to the image as an attestation, or it will generate an SBOM on the fly +by analyzing the contents of the image. + +The SBOM is cross-referenced with the [advisory database](/manuals/scout/deep-dive/advisory-db-sources.md) +to determine if any of the components in the image have known vulnerabilities. + +
+ +## Attestations + +{{< youtube-embed qOzcycbTs4o >}} + +[Build attestations](/manuals/build/metadata/attestations/_index.md) give you +detailed information about how an image was built and what it contains. These +attestations, generated by BuildKit during build-time, attach to the final +image as metadata, allowing you to inspect an image to see its origin, creator, +and contents. This information helps you make informed decisions about the +security and impact of the image on your supply chain. + +Docker Scout uses these attestations to evaluate the image's security and +supply chain posture, and to provide remediation recommendations for issues. If +issues are detected, such as missing or outdated attestations, Docker Scout can +guide you on how to add or update them, ensuring compliance and improving +visibility into the image's security status. + +There are two key types of attestations: + +- SBOM, which lists the software artifacts within the image. +- Provenance, which details how the image was built. + +You can create attestations by using `docker buildx build` with the +`--provenance` and `--sbom` flags. Attestations attach to the image index, +allowing you to inspect them without pulling the entire image. Docker Scout +leverages this metadata to give you more precise recommendations and better +control over your image's security. + +
+ +## Remediation + +{{< youtube-embed jM9zLBf8M-8 >}} + +Docker Scout's [remediation feature](/manuals/scout/policy/remediation.md) +helps you address supply chain and security issues by offering tailored +recommendations based on policy evaluations. These recommendations guide you in +improving policy compliance or enhancing image metadata, allowing Docker Scout +to perform more accurate evaluations in the future. + +You can use this feature to ensure that your base images are up-to-date and +that your supply chain attestations are complete. When a violation occurs, +Docker Scout provides recommended fixes, such as updating your base image or +adding missing attestations. If there isn’t enough information to determine +compliance, Docker Scout suggests actions to help resolve the issue. + +In the Docker Scout Dashboard, you can view and act on these recommendations by +reviewing violations or compliance uncertainties. With integrations like +GitHub, you can even automate updates, directly fixing issues from the +dashboard. + +
+ +## Common challenges and questions + + + +#### How is Docker Scout different from other security tools? + +Docker Scout takes a broader approach to container security compared to +third-party security tools. Third-party security tools, if they offer +remediation guidance at all, miss the mark on their limited scope of +application security posture within the software supply chain, and often +limited guidance when it comes to suggested fixes. Such tools have either +limitations on runtime monitoring or no runtime protection at all. When they do +offer runtime monitoring, it’s limited in its adherence to key policies. +Third-party security tools offer a limited scope of policy evaluation for +Docker-specific builds. By focusing on the entire software supply chain, +providing actionable guidance, and offering comprehensive runtime protection +with strong policy enforcement, Docker Scout goes beyond just identifying +vulnerabilities in your containers. It helps you build secure applications from +the ground up. + +#### Can I use Docker Scout with external registries other than Docker Hub? + +You can use Scout with registries other than Docker Hub. Integrating Docker Scout +with third-party container registries enables Docker Scout to run image +analysis on those repositories so that you can get insights into the +composition of those images even if they aren't hosted on Docker Hub. + +The following container registry integrations are available: + +- Artifactory +- Amazon Elastic Container Registry +- Azure Container Registry + +Learn more about configuring Scout with your registries in [Integrating Docker Scout with third-party registries](/scout/integrations/#container-registries). + +#### Does Docker Scout CLI come by default with Docker Desktop? + +Yes, the Docker Scout CLI plugin comes pre-installed with Docker Desktop. + +#### Is it possible to run `docker scout` commands on a Linux system without Docker Desktop? + +If you run Docker Engine without Docker Desktop, Docker Scout doesn't come +pre-installed, but you can [install it as a standalone binary](/scout/install/). + +#### How is Docker Scout using an SBOM? + +An SBOM, or software bill of materials, is a list of ingredients that make up +software components. [Docker Scout uses SBOMs](/scout/concepts/sbom/) to +determine the components that are used in a Docker image. When you analyze an +image, Docker Scout will either use the SBOM that is attached to the image (as +an attestation), or generate an SBOM on the fly by analyzing the contents of +the image. + +The SBOM is cross-referenced with the advisory database to determine if any of +the components in the image have known vulnerabilities. + +
diff --git a/content/guides/docker-scout/attestations.md b/content/guides/docker-scout/attestations.md deleted file mode 100644 index fb060e703363..000000000000 --- a/content/guides/docker-scout/attestations.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -title: Attestations -keywords: build, attestations, sbom, provenance, metadata -description: | - Introduction to SBOM and provenance attestations with Docker Build, - what they are, and why they exist -weight: 50 ---- - -{{< youtube-embed qOzcycbTs4o >}} - -[Build attestations](/manuals/build/metadata/attestations/_index.md) give you -detailed information about how an image was built and what it contains. These -attestations, generated by BuildKit during build-time, attach to the final -image as metadata, allowing you to inspect an image to see its origin, creator, -and contents. This information helps you make informed decisions about the -security and impact of the image on your supply chain. - -Docker Scout uses these attestations to evaluate the image's security and -supply chain posture, and to provide remediation recommendations for issues. If -issues are detected, such as missing or outdated attestations, Docker Scout can -guide you on how to add or update them, ensuring compliance and improving -visibility into the image's security status. - -There are two key types of attestations: - -- SBOM, which lists the software artifacts within the image. -- Provenance, which details how the image was built. - -You can create attestations by using `docker buildx build` with the -`--provenance` and `--sbom` flags. Attestations attach to the image index, -allowing you to inspect them without pulling the entire image. Docker Scout -leverages this metadata to give you more precise recommendations and better -control over your image's security. - -
diff --git a/content/guides/docker-scout/common-questions.md b/content/guides/docker-scout/common-questions.md deleted file mode 100644 index 4285d099d839..000000000000 --- a/content/guides/docker-scout/common-questions.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -title: Common challenges and questions -description: Explore common challenges and questions related to Docker Scout. -keywords: docker scout, faq, container security, vulnerability scanning, troubleshooting ---- - - - -### How is Docker Scout different from other security tools? - -Docker Scout takes a broader approach to container security compared to -third-party security tools. Third-party security tools, if they offer -remediation guidance at all, miss the mark on their limited scope of -application security posture within the software supply chain, and often -limited guidance when it comes to suggested fixes. Such tools have either -limitations on runtime monitoring or no runtime protection at all. When they do -offer runtime monitoring, it’s limited in its adherence to key policies. -Third-party security tools offer a limited scope of policy evaluation for -Docker-specific builds. By focusing on the entire software supply chain, -providing actionable guidance, and offering comprehensive runtime protection -with strong policy enforcement, Docker Scout goes beyond just identifying -vulnerabilities in your containers. It helps you build secure applications from -the ground up. - -### Can I use Docker Scout with external registries other than Docker Hub? - -You can use Scout with registries other than Docker Hub. Integrating Docker Scout -with third-party container registries enables Docker Scout to run image -analysis on those repositories so that you can get insights into the -composition of those images even if they aren't hosted on Docker Hub. - -The following container registry integrations are available: - -- Artifactory -- Amazon Elastic Container Registry -- Azure Container Registry - -Learn more about configuring Scout with your registries in [Integrating Docker Scout with third-party registries](/scout/integrations/#container-registries). - -### Does Docker Scout CLI come by default with Docker Desktop? - -Yes, the Docker Scout CLI plugin comes pre-installed with Docker Desktop. - -### Is it possible to run `docker scout` commands on a Linux system without Docker Desktop? - -If you run Docker Engine without Docker Desktop, Docker Scout doesn't come -pre-installed, but you can [install it as a standalone binary](/scout/install/). - -### How is Docker Scout using an SBOM? - -An SBOM, or software bill of materials, is a list of ingredients that make up -software components. [Docker Scout uses SBOMs](/scout/concepts/sbom/) to -determine the components that are used in a Docker image. When you analyze an -image, Docker Scout will either use the SBOM that is attached to the image (as -an attestation), or generate an SBOM on the fly by analyzing the contents of -the image. - -The SBOM is cross-referenced with the advisory database to determine if any of -the components in the image have known vulnerabilities. - -
diff --git a/content/guides/docker-scout/demo.md b/content/guides/docker-scout/demo.md deleted file mode 100644 index fb76b396aab9..000000000000 --- a/content/guides/docker-scout/demo.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -title: Docker Scout demo -linkTitle: Demo -description: Learn about Docker Scout's powerful features for enhanced supply chain security. -keywords: docker scout, demo, supply chain, vulnerability scanning, container security -weight: 20 ---- - -{{< youtube-embed "TkLwJ0p46W8" >}} - -Docker Scout has powerful features for enhancing containerized application -security and ensuring a robust software supply chain. - -- Define vulnerability remediation -- Discuss why remediation is essential to maintain the security and integrity - of containerized applications -- Discuss common vulnerabilities -- Implement remediation techniques: updating base images, applying patches, - removing unnecessary packages -- Verify and validate remediation efforts using Docker Scout - -
diff --git a/content/guides/docker-scout/remediation.md b/content/guides/docker-scout/remediation.md deleted file mode 100644 index e485c9cd3cf6..000000000000 --- a/content/guides/docker-scout/remediation.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -title: Remediation -description: Learn how Docker Scout can help you improve your software quality automatically, using remediation -keywords: scout, supply chain, security, remediation, automation -weight: 60 ---- - -{{< youtube-embed jM9zLBf8M-8 >}} - -Docker Scout's [remediation feature](/manuals/scout/policy/remediation.md) -helps you address supply chain and security issues by offering tailored -recommendations based on policy evaluations. These recommendations guide you in -improving policy compliance or enhancing image metadata, allowing Docker Scout -to perform more accurate evaluations in the future. - -You can use this feature to ensure that your base images are up-to-date and -that your supply chain attestations are complete. When a violation occurs, -Docker Scout provides recommended fixes, such as updating your base image or -adding missing attestations. If there isn’t enough information to determine -compliance, Docker Scout suggests actions to help resolve the issue. - -In the Docker Scout Dashboard, you can view and act on these recommendations by -reviewing violations or compliance uncertainties. With integrations like -GitHub, you can even automate updates, directly fixing issues from the -dashboard. - -
diff --git a/content/guides/docker-scout/s3c.md b/content/guides/docker-scout/s3c.md deleted file mode 100644 index df6eac67a7f2..000000000000 --- a/content/guides/docker-scout/s3c.md +++ /dev/null @@ -1,68 +0,0 @@ ---- -title: Software supply chain security -description: Learn about software supply chain security (S3C), what it means, and why it is important. -keywords: docker scout, secure, software, supply, chain, security, sssc, sscs, s3c -aliases: - - /scout/concepts/s3c/ -weight: 30 ---- - -{{< youtube-embed YzNK6E7APv0 >}} - -The term "software supply chain" refers to the end-to-end process of developing -and delivering software, from the development to deployment and maintenance. -Software supply chain security, or "S3C" for short, is the practice for -protecting the components and processes of the supply chain. - -S3C is a fundamental change in how organizations approach software security. -Traditionally in the software industry, security and compliance has been mostly -an afterthought, left to the software delivery or release phase. With S3C, -security is integrated into the entire software development lifecycle, from the -inner loop of development and testing, to the outer loop of shipping and -monitoring. - -Following industry best practices for software supply chain conduct is -important because it helps organizations protect their software from security -threats, compliance risks, and other vulnerabilities. Implementing a software -supply chain security framework improves visibility, collaboration, and -traceability of a project across stakeholders. This helps organizations detect, -respond to, and remediate threats more effectively. - -## Securing the software supply chain - -Building a secure software supply chain involves several key steps, such as: - -- Identify the software components and dependencies you use to build and run - your applications. -- Automate security testing throughout the software development lifecycle. -- Monitor your software supply chain for security threats. -- Implement security policies that govern how software is built, and the - components it contains. - -Managing the software supply chain is a complex task, especially in the modern -day where software is built using multiple components from different sources. -Organizations need to have a clear understanding of the software components -they use, and the security risks associated with them. - -## How Docker Scout is different - -Docker Scout is a platform designed to help organizations secure their software -supply chain. It provides tools and services for identifying and managing -software assets and policies, and automated remediation of security threats. - -Unlike traditional security tools that focus on scheduled, point-in-time scans -at specific stages in the software development lifecycle, Docker Scout uses a -modern event-driven model that spans the entire software supply chain. This -means that when a new vulnerability affecting your images is disclosed, your -updated risk assessment is available within seconds, and earlier in the -development process. - -Docker Scout works by analyzing the composition of your images to create a -Software Bill of Materials (SBOM). The SBOM is cross-referenced against the -security advisories to identify CVEs that affect your images. Docker Scout -integrates with [over 20 different security -advisories](/manuals/scout/deep-dive/advisory-db-sources.md), and updates its -vulnerability database in real-time. This ensures that your security posture is -represented using the latest available information. - -
diff --git a/content/guides/docker-scout/sbom.md b/content/guides/docker-scout/sbom.md deleted file mode 100644 index c7c6e2fa8a3a..000000000000 --- a/content/guides/docker-scout/sbom.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -title: Software Bill of Materials -description: Learn about Software Bill of Materials (SBOM) and how Docker Scout uses it. -keywords: scout, sbom, software bill of materials, analysis, composition -aliases: - - /scout/concepts/sbom/ -weight: 40 ---- - -{{< youtube-embed PbS4y7C7h4A >}} - -A Bill of Materials (BOM) is a list of materials, parts, and the quantities of -each needed to manufacture a product. For example, a BOM for a computer might -list the motherboard, CPU, RAM, power supply, storage devices, case, and other -components, along with the quantities of each that are needed to build the -computer. - -A Software Bill of Materials (SBOM) is a list of all the components that make -up a piece of software. This includes open source and third-party components, -as well as any custom code that has been written for the software. An SBOM is -similar to a BOM for a physical product, but for software. - -In the context of software supply chain security, SBOMs can help with -identifying and mitigating security and compliance risks in software. By -knowing exactly what components are used in a piece of software, you can -quickly identify and patch vulnerabilities in your components, or determine if -a component is licensed in a way that is incompatible with your project. - -## Contents of an SBOM - -An SBOM typically includes the following information: - -- The name of the software, such as the name of a library or framework, that - the SBOM describes. -- The version of the software. -- The license under which the software is distributed. -- A list of other components that the software depends on. - -## How Docker Scout uses SBOMs - -Docker Scout uses SBOMs to determine the components that are used in a Docker -image. When you analyze an image, Docker Scout will either use the SBOM that is -attached to the image as an attestation, or it will generate an SBOM on the fly -by analyzing the contents of the image. - -The SBOM is cross-referenced with the [advisory database](/manuals/scout/deep-dive/advisory-db-sources.md) -to determine if any of the components in the image have known vulnerabilities. - -
diff --git a/content/guides/docker-scout/why.md b/content/guides/docker-scout/why.md deleted file mode 100644 index 227702645b1c..000000000000 --- a/content/guides/docker-scout/why.md +++ /dev/null @@ -1,28 +0,0 @@ ---- -title: Why Docker Scout? -description: Learn how Docker Scout can help you secure your supply chain. -keywords: docker scout, supply chain security, vulnerability detection, sbom, container security -weight: 10 ---- - -{{< youtube-embed "-omsQ7Uqyc4" >}} - -Organizations face significant challenges from data breaches, -including financial losses, operational disruptions, and long-term damage to -brand reputation and customer trust. Docker Scout addresses critical problems -such as identifying insecure container images, preventing security breaches, -and reducing the risk of operational downtime due to vulnerabilities. - -Docker Scout provides several benefits: - -- Secure and trusted content -- A system of record for your Software Development Lifecycle (SDLC) -- Continuous security posture improvement - -Docker Scout offers automated vulnerability detection and remediation, helping -organizations identify and fix security issues in container images early in the -development process. It also integrates with popular development tools like -Docker Desktop and GitHub Actions, providing seamless security management and -compliance checks within existing workflows. - -
diff --git a/content/guides/dotnet/_index.md b/content/guides/dotnet/_index.md index 849c17f2b636..ea2e2e3bf0a2 100644 --- a/content/guides/dotnet/_index.md +++ b/content/guides/dotnet/_index.md @@ -8,13 +8,23 @@ keywords: getting started, .net aliases: - /language/dotnet/ - /guides/language/dotnet/ -languages: [c-sharp] + - /language/dotnet/develop/ + - /language/dotnet/run-tests/ + - /language/dotnet/configure-ci-cd/ + - /language/dotnet/deploy/ + - /guides/dotnet/configure-ci-cd/ + - /guides/dotnet/containerize/ + - /guides/dotnet/deploy/ + - /guides/dotnet/develop/ + - /guides/dotnet/run-tests/ params: + tags: [cicd] time: 20 minutes toc_min: 1 toc_max: 2 --- + The .NET getting started guide teaches you how to create a containerized .NET application using Docker. In this guide, you'll learn how to: - Containerize and run a .NET application @@ -26,3 +36,1271 @@ The .NET getting started guide teaches you how to create a containerized .NET ap After completing the .NET getting started modules, you should be able to containerize your own .NET application based on the examples and instructions provided in this guide. Start by containerizing an existing .NET application. + +## Containerize a .NET application + +### Prerequisites + +* You have installed the latest version of [Docker + Desktop](/get-started/get-docker.md). +* You have a [git client](https://git-scm.com/downloads). The examples in this + section use a command-line based git client, but you can use any client. + +### Overview + +This section walks you through containerizing and running a .NET +application. + +### Get the sample applications + +In this guide, you will use a pre-built .NET application. The application is +similar to the application built in the Docker Blog article, [Building a +Multi-Container .NET App Using Docker +Desktop](https://www.docker.com/blog/building-multi-container-net-app-using-docker-desktop/). + +Open a terminal, change directory to a directory that you want to work in, and +run the following command to clone the repository. + +```console +$ git clone https://github.com/docker/docker-dotnet-sample +``` + +### Create Docker assets + +Now that you have an application, you can create the necessary Docker assets to containerize it. You can choose between using the official .NET images or Docker Hardened Images (DHI). + +> [!TIP] +> +> [Gordon](/ai/gordon/), Docker's AI assistant, can generate Docker assets for your project. Ask Gordon to create a Dockerfile, Compose file, and `.dockerignore` tailored to your application. + +> [Docker Hardened Images (DHIs)](https://docs.docker.com/dhi/) are minimal, secure, and production-ready container base and application images maintained by Docker. DHI images are recommended for better security—they are designed to reduce vulnerabilities and simplify compliance. + +{{< tabs >}} +{{< tab name="Using Docker Hardened Images" >}} + +Docker Hardened Images (DHIs) for .NET are available in the [Docker Hardened Images catalog](https://hub.docker.com/hardened-images/catalog/dhi/aspnetcore). Docker Hardened Images are freely available to everyone with no subscription required. You can pull and use them like any other Docker image after signing in to the DHI registry. For more information, see the [DHI quickstart](/dhi/get-started/) guide. + +1. Sign in to the DHI registry: + ```console + $ docker login dhi.io + ``` + +2. Pull the .NET SDK DHI (check the catalog for available versions): + ```console + $ docker pull dhi.io/dotnet:10-sdk + ``` + +3. Pull the ASP.NET Core runtime DHI (check the catalog for available versions): + ```console + $ docker pull dhi.io/aspnetcore:10 + ``` + +Create the following files in your `docker-dotnet-sample` directory. + +```dockerfile {collapse=true,title=Dockerfile} +# syntax=docker/dockerfile:1 + +FROM --platform=$BUILDPLATFORM dhi.io/dotnet:10-sdk AS build +ARG TARGETARCH +COPY . /source +WORKDIR /source/src +RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages \ + dotnet publish -a ${TARGETARCH/amd64/x64} --use-current-runtime --self-contained false -o /app + +FROM dhi.io/aspnetcore:10 AS final +WORKDIR /app +COPY --from=build /app . +ENTRYPOINT ["dotnet", "myWebApp.dll"] +``` + +> [!NOTE] +> +> DHI runtime images already run as a non-root user (`nonroot`, UID 65532), so there's no need to create a user or specify `USER` in your Dockerfile. This reduces the attack surface and simplifies your configuration. + +```yaml {collapse=true,title=compose.yaml} +# Comments are provided throughout this file to help you get started. +# If you need more help, visit the Docker Compose reference guide at +# https://docs.docker.com/go/compose-spec-reference/ + +# Here the instructions define your application as a service called "server". +# This service is built from the Dockerfile in the current directory. +# You can add other services your application may depend on here, such as a +# database or a cache. For examples, see the Awesome Compose repository: +# https://github.com/docker/awesome-compose +services: + server: + build: + context: . + target: final + ports: + - 8080:8080 + +# The commented out section below is an example of how to define a PostgreSQL +# database that your application can use. `depends_on` tells Docker Compose to +# start the database before your application. The `db-data` volume persists the +# database data between container restarts. The `db-password` secret is used +# to set the database password. You must create `db/password.txt` and add +# a password of your choosing to it before running `docker compose up`. +# depends_on: +# db: +# condition: service_healthy +# db: +# image: postgres +# restart: always +# user: postgres +# secrets: +# - db-password +# volumes: +# - db-data:/var/lib/postgresql/data +# environment: +# - POSTGRES_DB=example +# - POSTGRES_PASSWORD_FILE=/run/secrets/db-password +# expose: +# - 5432 +# healthcheck: +# test: [ "CMD", "pg_isready" ] +# interval: 10s +# timeout: 5s +# retries: 5 +# volumes: +# db-data: +# secrets: +# db-password: +# file: db/password.txt +``` + +```text {collapse=true,title=".dockerignore"} +# Include any files or directories that you don't want to be copied to your +# container here (e.g., local build artifacts, temporary files, etc.). +# +# For more help, visit the .dockerignore file reference guide at +# https://docs.docker.com/go/build-context-dockerignore/ + +**/.classpath +**/.dockerignore +**/.env +**/.git +**/.gitignore +**/.project +**/.settings +**/.toolstarget +**/.vs +**/.vscode +**/*.*proj.user +**/*.dbmdl +**/*.jfm +**/bin +**/charts +**/docker-compose* +**/compose.y*ml +**/Dockerfile* +**/node_modules +**/npm-debug.log +**/obj +**/secrets.dev.yaml +**/values.dev.yaml +LICENSE +README.md +``` + +{{< /tab >}} +{{< tab name="Using the official .NET 10 image" >}} + +Create the following files in your `docker-dotnet-sample` directory. + +```dockerfile {collapse=true,title=Dockerfile} +# syntax=docker/dockerfile:1 + +FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:10.0-alpine AS build +ARG TARGETARCH +COPY . /source +WORKDIR /source/src +RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages \ + dotnet publish -a ${TARGETARCH/amd64/x64} --use-current-runtime --self-contained false -o /app + +FROM mcr.microsoft.com/dotnet/aspnet:10.0-alpine AS final +WORKDIR /app +COPY --from=build /app . +ARG UID=10001 +RUN adduser \ + --disabled-password \ + --gecos "" \ + --home "/nonexistent" \ + --shell "/sbin/nologin" \ + --no-create-home \ + --uid "${UID}" \ + appuser +USER appuser +ENTRYPOINT ["dotnet", "myWebApp.dll"] +``` + +```yaml {collapse=true,title=compose.yaml} +# Comments are provided throughout this file to help you get started. +# If you need more help, visit the Docker Compose reference guide at +# https://docs.docker.com/go/compose-spec-reference/ + +# Here the instructions define your application as a service called "server". +# This service is built from the Dockerfile in the current directory. +# You can add other services your application may depend on here, such as a +# database or a cache. For examples, see the Awesome Compose repository: +# https://github.com/docker/awesome-compose +services: + server: + build: + context: . + target: final + ports: + - 8080:8080 + +# The commented out section below is an example of how to define a PostgreSQL +# database that your application can use. `depends_on` tells Docker Compose to +# start the database before your application. The `db-data` volume persists the +# database data between container restarts. The `db-password` secret is used +# to set the database password. You must create `db/password.txt` and add +# a password of your choosing to it before running `docker compose up`. +# depends_on: +# db: +# condition: service_healthy +# db: +# image: postgres +# restart: always +# user: postgres +# secrets: +# - db-password +# volumes: +# - db-data:/var/lib/postgresql/data +# environment: +# - POSTGRES_DB=example +# - POSTGRES_PASSWORD_FILE=/run/secrets/db-password +# expose: +# - 5432 +# healthcheck: +# test: [ "CMD", "pg_isready" ] +# interval: 10s +# timeout: 5s +# retries: 5 +# volumes: +# db-data: +# secrets: +# db-password: +# file: db/password.txt +``` + +```text {collapse=true,title=".dockerignore"} +# Include any files or directories that you don't want to be copied to your +# container here (e.g., local build artifacts, temporary files, etc.). +# +# For more help, visit the .dockerignore file reference guide at +# https://docs.docker.com/go/build-context-dockerignore/ + +**/.classpath +**/.dockerignore +**/.env +**/.git +**/.gitignore +**/.project +**/.settings +**/.toolstarget +**/.vs +**/.vscode +**/*.*proj.user +**/*.dbmdl +**/*.jfm +**/bin +**/charts +**/docker-compose* +**/compose.y*ml +**/Dockerfile* +**/node_modules +**/npm-debug.log +**/obj +**/secrets.dev.yaml +**/values.dev.yaml +LICENSE +README.md +``` + +{{< /tab >}} +{{< /tabs >}} + +You should now have the following contents in your `docker-dotnet-sample` +directory. + +```text +├── docker-dotnet-sample/ +│ ├── .git/ +│ ├── src/ +│ ├── .dockerignore +│ ├── compose.yaml +│ ├── Dockerfile +│ └── README.md +``` + +To learn more about the files, see the following: + - [Dockerfile](/reference/dockerfile.md) + - [.dockerignore](/reference/dockerfile.md#dockerignore-file) + - [compose.yaml](/reference/compose-file/_index.md) + +### Run the application + +Inside the `docker-dotnet-sample` directory, run the following command in a +terminal. + +```console +$ docker compose up --build +``` + +Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see a simple web application. + +In the terminal, press `ctrl`+`c` to stop the application. + +#### Run the application in the background + +You can run the application detached from the terminal by adding the `-d` +option. Inside the `docker-dotnet-sample` directory, run the following command +in a terminal. + +```console +$ docker compose up --build -d +``` + +Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see a simple web application. + +In the terminal, run the following command to stop the application. + +```console +$ docker compose down +``` + +For more information about Compose commands, see the [Compose CLI +reference](/reference/cli/docker/compose/). + +### Summary + +In this section, you learned how you can containerize and run your .NET +application using Docker. + +Related information: + - [Dockerfile reference](/reference/dockerfile.md) + - [.dockerignore file reference](/reference/dockerfile.md#dockerignore-file) + - [Docker Compose overview](/manuals/compose/_index.md) + - [Docker Hardened Images](/dhi/) + +### Next steps + +In the next section, you'll learn how you can develop your application using +Docker containers. + +## Use containers for .NET development + +### Prerequisites + +Complete [Containerize a .NET application](containerize.md). + +### Overview + +In this section, you'll learn how to set up a development environment for your containerized application. This includes: + +- Adding a local database and persisting data +- Configuring Compose to automatically update your running Compose services as you edit and save your code +- Creating a development container that contains the .NET Core SDK tools and dependencies + +### Update the application + +This section uses a different branch of the `docker-dotnet-sample` repository +that contains an updated .NET application. The updated application is on the +`add-db` branch of the repository you cloned in [Containerize a .NET +application](containerize.md). + +To get the updated code, you need to checkout the `add-db` branch. For the changes you made in [Containerize a .NET application](containerize.md), for this section, you can stash them. In a terminal, run the following commands in the `docker-dotnet-sample` directory. + +1. Stash any previous changes. + + ```console + $ git stash -u + ``` + +2. Check out the new branch with the updated application. + + ```console + $ git checkout add-db + ``` + +In the `add-db` branch, only the .NET application has been updated. None of the Docker assets have been updated yet. + +You should now have the following in your `docker-dotnet-sample` directory. + +```text +├── docker-dotnet-sample/ +│ ├── .git/ +│ ├── src/ +│ │ ├── Data/ +│ │ ├── Models/ +│ │ ├── Pages/ +│ │ ├── Properties/ +│ │ ├── wwwroot/ +│ │ ├── appsettings.Development.json +│ │ ├── appsettings.json +│ │ ├── myWebApp.csproj +│ │ └── Program.cs +│ ├── tests/ +│ │ ├── tests.csproj +│ │ ├── UnitTest1.cs +│ │ └── Usings.cs +│ ├── .dockerignore +│ ├── .gitignore +│ ├── compose.yaml +│ ├── Dockerfile +│ └── README.md +``` + +### Add a local database and persist data + +You can use containers to set up local services, like a database. In this section, you'll update the `compose.yaml` file to define a database service and a volume to persist data. + +Open the `compose.yaml` file in an IDE or text editor. You'll notice it +already contains commented-out instructions for a PostgreSQL database and volume. + +Open `docker-dotnet-sample/src/appsettings.json` in an IDE or text editor. You'll +notice the connection string with all the database information. The +`compose.yaml` already contains this information, but it's commented out. +Uncomment the database instructions in the `compose.yaml` file. + +The following is the updated `compose.yaml` file. + +```yaml {hl_lines="8-33"} +services: + server: + build: + context: . + target: final + ports: + - 8080:8080 + depends_on: + db: + condition: service_healthy + db: + image: postgres:18 + restart: always + user: postgres + secrets: + - db-password + volumes: + - db-data:/var/lib/postgresql + environment: + - POSTGRES_DB=example + - POSTGRES_PASSWORD_FILE=/run/secrets/db-password + expose: + - 5432 + healthcheck: + test: ["CMD", "pg_isready"] + interval: 10s + timeout: 5s + retries: 5 +volumes: + db-data: +secrets: + db-password: + file: db/password.txt +``` + +> [!NOTE] +> +> To learn more about the instructions in the Compose file, see [Compose file +> reference](/reference/compose-file/). + +Before you run the application using Compose, notice that this Compose file uses +`secrets` and specifies a `password.txt` file to hold the database's password. +You must create this file as it's not included in the source repository. + +In the `docker-dotnet-sample` directory, create a new directory named `db` and +inside that directory create a file named `password.txt`. Open `password.txt` in an IDE or text editor and add the following password. The password must be on a single line, with no additional lines in the file. + +```text +example +``` + +Save and close the `password.txt` file. + +You should now have the following in your `docker-dotnet-sample` directory. + +```text +├── docker-dotnet-sample/ +│ ├── .git/ +│ ├── db/ +│ │ └── password.txt +│ ├── src/ +│ ├── tests/ +│ ├── .dockerignore +│ ├── .gitignore +│ ├── compose.yaml +│ ├── Dockerfile +│ └── README.md +``` + +Run the following command to start your application. + +```console +$ docker compose up --build +``` + +Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see a simple web application with the text `Student name is`. + +The application doesn't display a name because the database is empty. For this application, you need to access the database and then add records. + +### Add records to the database + +For the sample application, you must access the database directly to create sample records. + +You can run commands inside the database container using the `docker exec` +command. Before running that command, you must get the ID of the database +container. Open a new terminal window and run the following command to list all +your running containers. + +```console +$ docker container ls +``` + +You should see output like the following. + +```console +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +cb36e310aa7e docker-dotnet-sample-server "dotnet myWebApp.dll" About a minute ago Up About a minute 0.0.0.0:8080->8080/tcp docker-dotnet-sample-server-1 +39fdcf0aff7b postgres:18 "docker-entrypoint.s…" About a minute ago Up About a minute (healthy) 5432/tcp docker-dotnet-sample-db-1 +``` + +In the previous example, the container ID is `39fdcf0aff7b`. Run the following command to connect to the postgres database in the container. Replace the container ID with your own container ID. + +```console +$ docker exec -it 39fdcf0aff7b psql -d example -U postgres +``` + +And finally, insert a record into the database. + +```console +example=# INSERT INTO "Students" ("ID", "LastName", "FirstMidName", "EnrollmentDate") VALUES (DEFAULT, 'Whale', 'Moby', '2013-03-20'); +``` + +You should see output like the following. + +```console +INSERT 0 1 +``` + +Close the database connection and exit the container shell by running `exit`. + +```console +example=# exit +``` + +### Verify that data persists in the database + +Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see a simple web application with the text `Student name is Moby Whale`. + +Press `ctrl+c` in the terminal to stop your application. + +In the terminal, run `docker compose rm` to remove your containers and then run `docker compose up` to run your application again. + +```console +$ docker compose rm +$ docker compose up --build +``` + +Refresh [http://localhost:8080](http://localhost:8080) in your browser and verify that the student name persisted, even after the containers were removed and ran again. + +Press `ctrl+c` in the terminal to stop your application. + +### Automatically update services + +Use Compose Watch to automatically update your running Compose services as you edit and save your code. For more details about Compose Watch, see [Use Compose Watch](/manuals/compose/how-tos/file-watch.md). + +Open your `compose.yaml` file in an IDE or text editor and then add the Compose Watch instructions. The following is the updated `compose.yaml` file. + +```yaml {hl_lines="11-14"} +services: + server: + build: + context: . + target: final + ports: + - 8080:8080 + depends_on: + db: + condition: service_healthy + develop: + watch: + - action: rebuild + path: . + db: + image: postgres:18 + restart: always + user: postgres + secrets: + - db-password + volumes: + - db-data:/var/lib/postgresql + environment: + - POSTGRES_DB=example + - POSTGRES_PASSWORD_FILE=/run/secrets/db-password + expose: + - 5432 + healthcheck: + test: ["CMD", "pg_isready"] + interval: 10s + timeout: 5s + retries: 5 +volumes: + db-data: +secrets: + db-password: + file: db/password.txt +``` + +Run the following command to run your application with Compose Watch. + +```console +$ docker compose watch +``` + +Open a browser and verify that the application is running at [http://localhost:8080](http://localhost:8080). + +Any changes to the application's source files on your local machine will now be +immediately reflected in the running container. + +Open `docker-dotnet-sample/src/Pages/Index.cshtml` in an IDE or text editor and update the student name text on line 13 from `Student name is` to `Student name:`. + +```diff +-

Student name is @Model.StudentName

++

Student name: @Model.StudentName

+``` + +Save the changes to `Index.cshtml` and then wait a few seconds for the application to rebuild. Refresh [http://localhost:8080](http://localhost:8080) in your browser and verify that the updated text appears. + +Press `ctrl+c` in the terminal to stop your application. + +### Create a development container + +At this point, when you run your containerized application, it's using the .NET runtime image. While this small image is good for production, it lacks the SDK tools and dependencies you may need when developing. Also, during development, you may not need to run `dotnet publish`. You can use multi-stage builds to build stages for both development and production in the same Dockerfile. For more details, see [Multi-stage builds](/manuals/build/building/multi-stage.md). + +Add a new development stage to your Dockerfile and update your `compose.yaml` file to use this stage for local development. + +The following is the updated Dockerfile. + +{{< tabs >}} +{{< tab name="Using Docker Hardened Images" >}} + +```Dockerfile {hl_lines="10-13"} +# syntax=docker/dockerfile:1 + +FROM --platform=$BUILDPLATFORM dhi.io/dotnet:10-sdk AS build +ARG TARGETARCH +COPY . /source +WORKDIR /source/src +RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages \ + dotnet publish -a ${TARGETARCH/amd64/x64} --use-current-runtime --self-contained false -o /app + +FROM dhi.io/dotnet:10-sdk AS development +COPY . /source +WORKDIR /source/src +CMD dotnet run --no-launch-profile + +FROM dhi.io/aspnetcore:10 AS final +WORKDIR /app +COPY --from=build /app . +ENTRYPOINT ["dotnet", "myWebApp.dll"] +``` + +{{< /tab >}} +{{< tab name="Using the official .NET 10 image" >}} + +```Dockerfile {hl_lines="10-13"} +# syntax=docker/dockerfile:1 + +FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:10.0-alpine AS build +ARG TARGETARCH +COPY . /source +WORKDIR /source/src +RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages \ + dotnet publish -a ${TARGETARCH/amd64/x64} --use-current-runtime --self-contained false -o /app + +FROM mcr.microsoft.com/dotnet/sdk:10.0-alpine AS development +COPY . /source +WORKDIR /source/src +CMD dotnet run --no-launch-profile + +FROM mcr.microsoft.com/dotnet/aspnet:10.0-alpine AS final +WORKDIR /app +COPY --from=build /app . +ARG UID=10001 +RUN adduser \ + --disabled-password \ + --gecos "" \ + --home "/nonexistent" \ + --shell "/sbin/nologin" \ + --no-create-home \ + --uid "${UID}" \ + appuser +USER appuser +ENTRYPOINT ["dotnet", "myWebApp.dll"] +``` + +{{< /tab >}} +{{< /tabs >}} + +The following is the updated `compose.yaml` file. + +```yaml {hl_lines=[5,15,16]} +services: + server: + build: + context: . + target: development + ports: + - 8080:8080 + depends_on: + db: + condition: service_healthy + develop: + watch: + - action: rebuild + path: . + environment: + - ASPNETCORE_ENVIRONMENT=Development + db: + image: postgres:18 + restart: always + user: postgres + secrets: + - db-password + volumes: + - db-data:/var/lib/postgresql + environment: + - POSTGRES_DB=example + - POSTGRES_PASSWORD_FILE=/run/secrets/db-password + expose: + - 5432 + healthcheck: + test: ["CMD", "pg_isready"] + interval: 10s + timeout: 5s + retries: 5 +volumes: + db-data: +secrets: + db-password: + file: db/password.txt +``` + +Your containerized application will now use the SDK image (either `dhi.io/dotnet:10-sdk` for DHI or `mcr.microsoft.com/dotnet/sdk:10.0-alpine` for official images), which includes development tools like `dotnet test`. Continue to the next section to learn how you can run `dotnet test`. + +### Summary + +In this section, you took a look at setting up your Compose file to add a local +database and persist data. You also learned how to use Compose Watch to automatically rebuild and run your container when you update your code. And finally, you learned how to create a development container that contains the SDK tools and dependencies needed for development. + +Related information: + +- [Compose file reference](/reference/compose-file/) +- [Compose file watch](/manuals/compose/how-tos/file-watch.md) +- [Multi-stage builds](/manuals/build/building/multi-stage.md) + +### Next steps + +In the next section, you'll learn how to run unit tests using Docker. + +## Run .NET tests in a container + +### Prerequisites + +Complete all the previous sections of this guide, starting with [Containerize a .NET application](containerize.md). + +### Overview + +Testing is an essential part of modern software development. Testing can mean a +lot of things to different development teams. There are unit tests, integration +tests and end-to-end testing. In this guide you take a look at running your unit +tests in Docker when developing and when building. + +### Run tests when developing locally + +The sample application already has an xUnit test inside the `tests` directory. When developing locally, you can use Compose to run your tests. + +Run the following command in the `docker-dotnet-sample` directory to run the tests inside a container. + +```console +$ docker compose run --build --rm server dotnet test /source/tests +``` + +You should see output that contains the following. + +```console +Starting test execution, please wait... +A total of 1 test files matched the specified pattern. + +Passed! - Failed: 0, Passed: 1, Skipped: 0, Total: 1, Duration: < 1 ms - /source/tests/bin/Debug/net10.0/tests.dll (net10.0) +``` + +To learn more about the command, see [docker compose run](/reference/cli/docker/compose/run/). + +### Run tests when building + +To run your tests when building, you need to update your Dockerfile. You can create a new test stage that runs the tests, or run the tests in the existing build stage. For this guide, update the Dockerfile to run the tests in the build stage. + +The following is the updated Dockerfile. + +{{< tabs >}} +{{< tab name="Using Docker Hardened Images" >}} + +```dockerfile {hl_lines="9"} +# syntax=docker/dockerfile:1 + +FROM --platform=$BUILDPLATFORM dhi.io/dotnet:10-sdk AS build +ARG TARGETARCH +COPY . /source +WORKDIR /source/src +RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages \ + dotnet publish -a ${TARGETARCH/amd64/x64} --use-current-runtime --self-contained false -o /app +RUN dotnet test /source/tests + +FROM dhi.io/dotnet:10-sdk AS development +COPY . /source +WORKDIR /source/src +CMD dotnet run --no-launch-profile + +FROM dhi.io/aspnetcore:10 AS final +WORKDIR /app +COPY --from=build /app . +ENTRYPOINT ["dotnet", "myWebApp.dll"] +``` + +{{< /tab >}} +{{< tab name="Using the official .NET 10 image" >}} + +```dockerfile {hl_lines="9"} +# syntax=docker/dockerfile:1 + +FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:10.0-alpine AS build +ARG TARGETARCH +COPY . /source +WORKDIR /source/src +RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages \ + dotnet publish -a ${TARGETARCH/amd64/x64} --use-current-runtime --self-contained false -o /app +RUN dotnet test /source/tests + +FROM mcr.microsoft.com/dotnet/sdk:10.0-alpine AS development +COPY . /source +WORKDIR /source/src +CMD dotnet run --no-launch-profile + +FROM mcr.microsoft.com/dotnet/aspnet:10.0-alpine AS final +WORKDIR /app +COPY --from=build /app . +ARG UID=10001 +RUN adduser \ + --disabled-password \ + --gecos "" \ + --home "/nonexistent" \ + --shell "/sbin/nologin" \ + --no-create-home \ + --uid "${UID}" \ + appuser +USER appuser +ENTRYPOINT ["dotnet", "myWebApp.dll"] +``` + +{{< /tab >}} +{{< /tabs >}} + +Run the following command to build an image using the build stage as the target and view the test results. Include `--progress=plain` to view the build output, `--no-cache` to ensure the tests always run, and `--target build` to target the build stage. + +```console +$ docker build -t dotnet-docker-image-test --progress=plain --no-cache --target build . +``` + +You should see output containing the following. + +```console +#11 [build 5/5] RUN dotnet test /source/tests +#11 1.564 Determining projects to restore... +#11 3.421 Restored /source/src/myWebApp.csproj (in 1.02 sec). +#11 19.42 Restored /source/tests/tests.csproj (in 17.05 sec). +#11 27.91 myWebApp -> /source/src/bin/Debug/net10.0/myWebApp.dll +#11 28.47 tests -> /source/tests/bin/Debug/net10.0/tests.dll +#11 28.49 Test run for /source/tests/bin/Debug/net10.0/tests.dll (.NETCoreApp,Version=v10.0) +#11 28.67 Microsoft (R) Test Execution Command Line Tool Version 17.3.3 (x64) +#11 28.67 Copyright (c) Microsoft Corporation. All rights reserved. +#11 28.68 +#11 28.97 Starting test execution, please wait... +#11 29.03 A total of 1 test files matched the specified pattern. +#11 32.07 +#11 32.08 Passed! - Failed: 0, Passed: 1, Skipped: 0, Total: 1, Duration: < 1 ms - /source/tests/bin/Debug/net10.0/tests.dll (net10.0) +#11 DONE 32.2s +``` + +### Summary + +In this section, you learned how to run tests when developing locally using Compose and how to run tests when building your image. + +Related information: + +- [docker compose run](/reference/cli/docker/compose/run/) + +### Next steps + +Next, you’ll learn how to set up a CI/CD pipeline using GitHub Actions. + +## Configure CI/CD for your .NET application + +### Prerequisites + +Complete all the previous sections of this guide, starting with [Containerize a .NET application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. + +### Overview + +In this section, you'll learn how to set up and use GitHub Actions to build and test your Docker image as well as push it to Docker Hub. You will complete the following steps: + +1. Create a new repository on GitHub. +2. Define the GitHub Actions workflow. +3. Run the workflow. + +### Step one: Create the repository + +Create a GitHub repository, configure the Docker Hub credentials, and push your source code. + +1. [Create a new repository](https://github.com/new) on GitHub. + +2. Open the repository **Settings**, and go to **Secrets and variables** > + **Actions**. + +3. Create a new **Repository variable** named `DOCKER_USERNAME` and your Docker ID as a value. + +4. Create a new [Personal Access Token (PAT)](/manuals/security/access-tokens.md#create-an-access-token) for Docker Hub. You can name this token `docker-tutorial`. Make sure access permissions include Read and Write. + +5. Add the PAT as a **Repository secret** in your GitHub repository, with the name + `DOCKERHUB_TOKEN`. + +6. In your local repository on your machine, run the following command to change + the origin to the repository you just created. Make sure you change + `your-username` to your GitHub username and `your-repository` to the name of + the repository you created. + + ```console + $ git remote set-url origin https://github.com/your-username/your-repository.git + ``` + +7. In your local repository on your machine, run the following command to rename + the branch to main. + + ```console + $ git branch -M main + ``` + +8. Run the following commands to stage, commit, and then push your local + repository to GitHub. + + ```console + $ git add -A + $ git commit -m "my first commit" + $ git push -u origin main + ``` + +### Step two: Set up the workflow + +Set up your GitHub Actions workflow for building, testing, and pushing the image +to Docker Hub. + +1. Go to your repository on GitHub and then select the **Actions** tab. + +2. Select **set up a workflow yourself**. + + This takes you to a page for creating a new GitHub actions workflow file in + your repository, under `.github/workflows/main.yml` by default. + +3. In the editor window, copy and paste the following YAML configuration. + + ```yaml + name: ci + + on: + push: + branches: + - main + + jobs: + build: + runs-on: ubuntu-latest + steps: + - name: Login to Docker Hub + uses: docker/login-action@{{% param "login_action_version" %}} + with: + username: ${{ vars.DOCKER_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} + + - name: Build and test + uses: docker/build-push-action@{{% param "build_push_action_version" %}} + with: + target: build + load: true + + - name: Build and push + uses: docker/build-push-action@{{% param "build_push_action_version" %}} + with: + platforms: linux/amd64,linux/arm64 + push: true + target: final + tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest + ``` + + For more information about the YAML syntax for `docker/build-push-action`, + refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). + +### Step three: Run the workflow + +Save the workflow file and run the job. + +1. Select **Commit changes...** and push the changes to the `main` branch. + + After pushing the commit, the workflow starts automatically. + +2. Go to the **Actions** tab. It displays the workflow. + + Selecting the workflow shows you the breakdown of all the steps. + +3. When the workflow is complete, go to your + [repositories on Docker Hub](https://hub.docker.com/repositories). + + If you see the new repository in that list, it means the GitHub Actions + successfully pushed the image to Docker Hub. + +### Summary + +In this section, you learned how to set up a GitHub Actions workflow for your application. + +Related information: + +- [Introduction to GitHub Actions](/guides/gha.md) +- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) +- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) + +### Next steps + +Next, learn how you can locally test and debug your workloads on Kubernetes before deploying. + +## Test your .NET deployment + +### Prerequisites + +- Complete all the previous sections of this guide, starting with [Containerize + a .NET application](containerize.md). +- [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker + Desktop. + +### Overview + +In this section, you'll learn how to use Docker Desktop to deploy your +application to a fully-featured Kubernetes environment on your development +machine. This allows you to test and debug your workloads on Kubernetes locally +before deploying. + +### Create a Kubernetes YAML file + +In your `docker-dotnet-sample` directory, create a file named +`docker-dotnet-kubernetes.yaml`. Open the file in an IDE or text editor and add +the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker +username and the name of the repository that you created in [Configure CI/CD for +your .NET application](configure-ci-cd.md). + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + labels: + service: server + name: server + namespace: default +spec: + replicas: 1 + selector: + matchLabels: + service: server + strategy: {} + template: + metadata: + labels: + service: server + spec: + initContainers: + - name: wait-for-db + image: busybox:1.28 + command: + [ + "sh", + "-c", + 'until nc -zv db 5432; do echo "waiting for db"; sleep 2; done;', + ] + containers: + - image: DOCKER_USERNAME/REPO_NAME + name: server + imagePullPolicy: Always + ports: + - containerPort: 8080 + hostPort: 8080 + protocol: TCP + resources: {} + restartPolicy: Always +status: {} +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + labels: + service: db + name: db + namespace: default +spec: + replicas: 1 + selector: + matchLabels: + service: db + strategy: + type: Recreate + template: + metadata: + labels: + service: db + spec: + containers: + - env: + - name: POSTGRES_DB + value: example + - name: POSTGRES_PASSWORD + value: example + image: postgres:18 + name: db + ports: + - containerPort: 5432 + protocol: TCP + resources: {} + restartPolicy: Always +status: {} +--- +apiVersion: v1 +kind: Service +metadata: + labels: + service: server + name: server + namespace: default +spec: + type: NodePort + ports: + - name: "8080" + port: 8080 + targetPort: 8080 + nodePort: 30001 + selector: + service: server +status: + loadBalancer: {} +--- +apiVersion: v1 +kind: Service +metadata: + labels: + service: db + name: db + namespace: default +spec: + ports: + - name: "5432" + port: 5432 + targetPort: 5432 + selector: + service: db +status: + loadBalancer: {} +``` + +In this Kubernetes YAML file, there are four objects, separated by the `---`. In addition to a Service and Deployment for the database, the other two objects are: + +- A Deployment, describing a scalable group of identical pods. In this case, + you'll get just one replica, or copy of your pod. That pod, which is + described under `template`, has just one container in it. The container is + created from the image built by GitHub Actions in [Configure CI/CD for your + .NET application](configure-ci-cd.md). +- A NodePort service, which will route traffic from port 30001 on your host to + port 8080 inside the pods it routes to, allowing you to reach your app + from the network. + +To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). + +### Deploy and check your application + +1. In a terminal, navigate to the `docker-dotnet-sample` directory + and deploy your application to Kubernetes. + + ```console + $ kubectl apply -f docker-dotnet-kubernetes.yaml + ``` + + You should see output that looks like the following, indicating your Kubernetes objects were created successfully. + + ```shell + deployment.apps/db created + service/db created + deployment.apps/server created + service/server created + ``` + +2. Make sure everything worked by listing your deployments. + + ```console + $ kubectl get deployments + ``` + + Your deployment should be listed as follows: + + ```shell + NAME READY UP-TO-DATE AVAILABLE AGE + db 1/1 1 1 76s + server 1/1 1 1 76s + ``` + + This indicates all of the pods are up and running. Do the same check for your services. + + ```console + $ kubectl get services + ``` + + You should get output like the following. + + ```shell + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + db ClusterIP 10.96.156.90 5432/TCP 2m8s + kubernetes ClusterIP 10.96.0.1 443/TCP 164m + server NodePort 10.102.94.225 8080:30001/TCP 2m8s + ``` + + In addition to the default `kubernetes` service, you can see your `server` service and `db` service. The `server` service is accepting traffic on port 30001/TCP. + +3. Open a browser and visit your app at `localhost:30001`. You should see your + application. + +4. Run the following command to tear down your application. + + ```console + $ kubectl delete -f docker-dotnet-kubernetes.yaml + ``` + +### Summary + +In this section, you learned how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. + +Related information: + +- [Kubernetes documentation](https://kubernetes.io/docs/home/) +- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) +- [Swarm mode overview](/manuals/engine/swarm/_index.md) diff --git a/content/guides/dotnet/configure-ci-cd.md b/content/guides/dotnet/configure-ci-cd.md deleted file mode 100644 index b5065330ef9f..000000000000 --- a/content/guides/dotnet/configure-ci-cd.md +++ /dev/null @@ -1,148 +0,0 @@ ---- -title: Configure CI/CD for your .NET application -linkTitle: Configure CI/CD -weight: 40 -keywords: .net, CI/CD -description: Learn how to Configure CI/CD for your .NET application -aliases: - - /language/dotnet/configure-ci-cd/ - - /guides/language/dotnet/configure-ci-cd/ ---- - -## Prerequisites - -Complete all the previous sections of this guide, starting with [Containerize a .NET application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. - -## Overview - -In this section, you'll learn how to set up and use GitHub Actions to build and test your Docker image as well as push it to Docker Hub. You will complete the following steps: - -1. Create a new repository on GitHub. -2. Define the GitHub Actions workflow. -3. Run the workflow. - -## Step one: Create the repository - -Create a GitHub repository, configure the Docker Hub credentials, and push your source code. - -1. [Create a new repository](https://github.com/new) on GitHub. - -2. Open the repository **Settings**, and go to **Secrets and variables** > - **Actions**. - -3. Create a new **Repository variable** named `DOCKER_USERNAME` and your Docker ID as a value. - -4. Create a new [Personal Access Token (PAT)](/manuals/security/access-tokens.md#create-an-access-token) for Docker Hub. You can name this token `docker-tutorial`. Make sure access permissions include Read and Write. - -5. Add the PAT as a **Repository secret** in your GitHub repository, with the name - `DOCKERHUB_TOKEN`. - -6. In your local repository on your machine, run the following command to change - the origin to the repository you just created. Make sure you change - `your-username` to your GitHub username and `your-repository` to the name of - the repository you created. - - ```console - $ git remote set-url origin https://github.com/your-username/your-repository.git - ``` - -7. In your local repository on your machine, run the following command to rename - the branch to main. - - ```console - $ git branch -M main - ``` - -8. Run the following commands to stage, commit, and then push your local - repository to GitHub. - - ```console - $ git add -A - $ git commit -m "my first commit" - $ git push -u origin main - ``` - -## Step two: Set up the workflow - -Set up your GitHub Actions workflow for building, testing, and pushing the image -to Docker Hub. - -1. Go to your repository on GitHub and then select the **Actions** tab. - -2. Select **set up a workflow yourself**. - - This takes you to a page for creating a new GitHub actions workflow file in - your repository, under `.github/workflows/main.yml` by default. - -3. In the editor window, copy and paste the following YAML configuration. - - ```yaml - name: ci - - on: - push: - branches: - - main - - jobs: - build: - runs-on: ubuntu-latest - steps: - - name: Login to Docker Hub - uses: docker/login-action@{{% param "login_action_version" %}} - with: - username: ${{ vars.DOCKER_USERNAME }} - password: ${{ secrets.DOCKERHUB_TOKEN }} - - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} - - - name: Build and test - uses: docker/build-push-action@{{% param "build_push_action_version" %}} - with: - target: build - load: true - - - name: Build and push - uses: docker/build-push-action@{{% param "build_push_action_version" %}} - with: - platforms: linux/amd64,linux/arm64 - push: true - target: final - tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest - ``` - - For more information about the YAML syntax for `docker/build-push-action`, - refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). - -## Step three: Run the workflow - -Save the workflow file and run the job. - -1. Select **Commit changes...** and push the changes to the `main` branch. - - After pushing the commit, the workflow starts automatically. - -2. Go to the **Actions** tab. It displays the workflow. - - Selecting the workflow shows you the breakdown of all the steps. - -3. When the workflow is complete, go to your - [repositories on Docker Hub](https://hub.docker.com/repositories). - - If you see the new repository in that list, it means the GitHub Actions - successfully pushed the image to Docker Hub. - -## Summary - -In this section, you learned how to set up a GitHub Actions workflow for your application. - -Related information: - -- [Introduction to GitHub Actions](/guides/gha.md) -- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) -- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) - -## Next steps - -Next, learn how you can locally test and debug your workloads on Kubernetes before deploying. diff --git a/content/guides/dotnet/containerize.md b/content/guides/dotnet/containerize.md deleted file mode 100644 index d5126324f1dd..000000000000 --- a/content/guides/dotnet/containerize.md +++ /dev/null @@ -1,364 +0,0 @@ ---- -title: Containerize a .NET application -linkTitle: Containerize your app -weight: 10 -keywords: .net, containerize, initialize -description: Learn how to containerize an ASP.NET application. -aliases: -- /language/dotnet/build-images/ -- /language/dotnet/run-containers/ -- /language/dotnet/containerize/ -- /guides/language/dotnet/containerize/ ---- - -## Prerequisites - -* You have installed the latest version of [Docker - Desktop](/get-started/get-docker.md). -* You have a [git client](https://git-scm.com/downloads). The examples in this - section use a command-line based git client, but you can use any client. - -## Overview - -This section walks you through containerizing and running a .NET -application. - -## Get the sample applications - -In this guide, you will use a pre-built .NET application. The application is -similar to the application built in the Docker Blog article, [Building a -Multi-Container .NET App Using Docker -Desktop](https://www.docker.com/blog/building-multi-container-net-app-using-docker-desktop/). - -Open a terminal, change directory to a directory that you want to work in, and -run the following command to clone the repository. - -```console -$ git clone https://github.com/docker/docker-dotnet-sample -``` - -## Create Docker assets - -Now that you have an application, you can create the necessary Docker assets to containerize it. You can choose between using the official .NET images or Docker Hardened Images (DHI). - -> [!TIP] -> -> [Gordon](/ai/gordon/), Docker's AI assistant, can generate Docker assets for your project. Ask Gordon to create a Dockerfile, Compose file, and `.dockerignore` tailored to your application. - -> [Docker Hardened Images (DHIs)](https://docs.docker.com/dhi/) are minimal, secure, and production-ready container base and application images maintained by Docker. DHI images are recommended for better security—they are designed to reduce vulnerabilities and simplify compliance. - -{{< tabs >}} -{{< tab name="Using Docker Hardened Images" >}} - -Docker Hardened Images (DHIs) for .NET are available in the [Docker Hardened Images catalog](https://hub.docker.com/hardened-images/catalog/dhi/aspnetcore). Docker Hardened Images are freely available to everyone with no subscription required. You can pull and use them like any other Docker image after signing in to the DHI registry. For more information, see the [DHI quickstart](/dhi/get-started/) guide. - -1. Sign in to the DHI registry: - ```console - $ docker login dhi.io - ``` - -2. Pull the .NET SDK DHI (check the catalog for available versions): - ```console - $ docker pull dhi.io/dotnet:10-sdk - ``` - -3. Pull the ASP.NET Core runtime DHI (check the catalog for available versions): - ```console - $ docker pull dhi.io/aspnetcore:10 - ``` - -Create the following files in your `docker-dotnet-sample` directory. - -```dockerfile {collapse=true,title=Dockerfile} -# syntax=docker/dockerfile:1 - -FROM --platform=$BUILDPLATFORM dhi.io/dotnet:10-sdk AS build -ARG TARGETARCH -COPY . /source -WORKDIR /source/src -RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages \ - dotnet publish -a ${TARGETARCH/amd64/x64} --use-current-runtime --self-contained false -o /app - -FROM dhi.io/aspnetcore:10 AS final -WORKDIR /app -COPY --from=build /app . -ENTRYPOINT ["dotnet", "myWebApp.dll"] -``` - -> [!NOTE] -> -> DHI runtime images already run as a non-root user (`nonroot`, UID 65532), so there's no need to create a user or specify `USER` in your Dockerfile. This reduces the attack surface and simplifies your configuration. - -```yaml {collapse=true,title=compose.yaml} -# Comments are provided throughout this file to help you get started. -# If you need more help, visit the Docker Compose reference guide at -# https://docs.docker.com/go/compose-spec-reference/ - -# Here the instructions define your application as a service called "server". -# This service is built from the Dockerfile in the current directory. -# You can add other services your application may depend on here, such as a -# database or a cache. For examples, see the Awesome Compose repository: -# https://github.com/docker/awesome-compose -services: - server: - build: - context: . - target: final - ports: - - 8080:8080 - -# The commented out section below is an example of how to define a PostgreSQL -# database that your application can use. `depends_on` tells Docker Compose to -# start the database before your application. The `db-data` volume persists the -# database data between container restarts. The `db-password` secret is used -# to set the database password. You must create `db/password.txt` and add -# a password of your choosing to it before running `docker compose up`. -# depends_on: -# db: -# condition: service_healthy -# db: -# image: postgres -# restart: always -# user: postgres -# secrets: -# - db-password -# volumes: -# - db-data:/var/lib/postgresql/data -# environment: -# - POSTGRES_DB=example -# - POSTGRES_PASSWORD_FILE=/run/secrets/db-password -# expose: -# - 5432 -# healthcheck: -# test: [ "CMD", "pg_isready" ] -# interval: 10s -# timeout: 5s -# retries: 5 -# volumes: -# db-data: -# secrets: -# db-password: -# file: db/password.txt -``` - -```text {collapse=true,title=".dockerignore"} -# Include any files or directories that you don't want to be copied to your -# container here (e.g., local build artifacts, temporary files, etc.). -# -# For more help, visit the .dockerignore file reference guide at -# https://docs.docker.com/go/build-context-dockerignore/ - -**/.classpath -**/.dockerignore -**/.env -**/.git -**/.gitignore -**/.project -**/.settings -**/.toolstarget -**/.vs -**/.vscode -**/*.*proj.user -**/*.dbmdl -**/*.jfm -**/bin -**/charts -**/docker-compose* -**/compose.y*ml -**/Dockerfile* -**/node_modules -**/npm-debug.log -**/obj -**/secrets.dev.yaml -**/values.dev.yaml -LICENSE -README.md -``` - -{{< /tab >}} -{{< tab name="Using the official .NET 10 image" >}} - -Create the following files in your `docker-dotnet-sample` directory. - -```dockerfile {collapse=true,title=Dockerfile} -# syntax=docker/dockerfile:1 - -FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:10.0-alpine AS build -ARG TARGETARCH -COPY . /source -WORKDIR /source/src -RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages \ - dotnet publish -a ${TARGETARCH/amd64/x64} --use-current-runtime --self-contained false -o /app - -FROM mcr.microsoft.com/dotnet/aspnet:10.0-alpine AS final -WORKDIR /app -COPY --from=build /app . -ARG UID=10001 -RUN adduser \ - --disabled-password \ - --gecos "" \ - --home "/nonexistent" \ - --shell "/sbin/nologin" \ - --no-create-home \ - --uid "${UID}" \ - appuser -USER appuser -ENTRYPOINT ["dotnet", "myWebApp.dll"] -``` - -```yaml {collapse=true,title=compose.yaml} -# Comments are provided throughout this file to help you get started. -# If you need more help, visit the Docker Compose reference guide at -# https://docs.docker.com/go/compose-spec-reference/ - -# Here the instructions define your application as a service called "server". -# This service is built from the Dockerfile in the current directory. -# You can add other services your application may depend on here, such as a -# database or a cache. For examples, see the Awesome Compose repository: -# https://github.com/docker/awesome-compose -services: - server: - build: - context: . - target: final - ports: - - 8080:8080 - -# The commented out section below is an example of how to define a PostgreSQL -# database that your application can use. `depends_on` tells Docker Compose to -# start the database before your application. The `db-data` volume persists the -# database data between container restarts. The `db-password` secret is used -# to set the database password. You must create `db/password.txt` and add -# a password of your choosing to it before running `docker compose up`. -# depends_on: -# db: -# condition: service_healthy -# db: -# image: postgres -# restart: always -# user: postgres -# secrets: -# - db-password -# volumes: -# - db-data:/var/lib/postgresql/data -# environment: -# - POSTGRES_DB=example -# - POSTGRES_PASSWORD_FILE=/run/secrets/db-password -# expose: -# - 5432 -# healthcheck: -# test: [ "CMD", "pg_isready" ] -# interval: 10s -# timeout: 5s -# retries: 5 -# volumes: -# db-data: -# secrets: -# db-password: -# file: db/password.txt -``` - -```text {collapse=true,title=".dockerignore"} -# Include any files or directories that you don't want to be copied to your -# container here (e.g., local build artifacts, temporary files, etc.). -# -# For more help, visit the .dockerignore file reference guide at -# https://docs.docker.com/go/build-context-dockerignore/ - -**/.classpath -**/.dockerignore -**/.env -**/.git -**/.gitignore -**/.project -**/.settings -**/.toolstarget -**/.vs -**/.vscode -**/*.*proj.user -**/*.dbmdl -**/*.jfm -**/bin -**/charts -**/docker-compose* -**/compose.y*ml -**/Dockerfile* -**/node_modules -**/npm-debug.log -**/obj -**/secrets.dev.yaml -**/values.dev.yaml -LICENSE -README.md -``` - -{{< /tab >}} -{{< /tabs >}} - -You should now have the following contents in your `docker-dotnet-sample` -directory. - -```text -├── docker-dotnet-sample/ -│ ├── .git/ -│ ├── src/ -│ ├── .dockerignore -│ ├── compose.yaml -│ ├── Dockerfile -│ └── README.md -``` - -To learn more about the files, see the following: - - [Dockerfile](/reference/dockerfile.md) - - [.dockerignore](/reference/dockerfile.md#dockerignore-file) - - [compose.yaml](/reference/compose-file/_index.md) - -## Run the application - -Inside the `docker-dotnet-sample` directory, run the following command in a -terminal. - -```console -$ docker compose up --build -``` - -Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see a simple web application. - -In the terminal, press `ctrl`+`c` to stop the application. - -### Run the application in the background - -You can run the application detached from the terminal by adding the `-d` -option. Inside the `docker-dotnet-sample` directory, run the following command -in a terminal. - -```console -$ docker compose up --build -d -``` - -Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see a simple web application. - -In the terminal, run the following command to stop the application. - -```console -$ docker compose down -``` - -For more information about Compose commands, see the [Compose CLI -reference](/reference/cli/docker/compose/). - -## Summary - -In this section, you learned how you can containerize and run your .NET -application using Docker. - -Related information: - - [Dockerfile reference](/reference/dockerfile.md) - - [.dockerignore file reference](/reference/dockerfile.md#dockerignore-file) - - [Docker Compose overview](/manuals/compose/_index.md) - - [Docker Hardened Images](/dhi/) - -## Next steps - -In the next section, you'll learn how you can develop your application using -Docker containers. diff --git a/content/guides/dotnet/deploy.md b/content/guides/dotnet/deploy.md deleted file mode 100644 index 0f4089ab88c5..000000000000 --- a/content/guides/dotnet/deploy.md +++ /dev/null @@ -1,224 +0,0 @@ ---- -title: Test your .NET deployment -linkTitle: Test your deployment -weight: 50 -keywords: deploy, .net, local, development -description: Learn how to deploy your application -aliases: - - /language/dotnet/deploy/ - - /guides/language/dotnet/deploy/ ---- - -## Prerequisites - -- Complete all the previous sections of this guide, starting with [Containerize - a .NET application](containerize.md). -- [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker - Desktop. - -## Overview - -In this section, you'll learn how to use Docker Desktop to deploy your -application to a fully-featured Kubernetes environment on your development -machine. This allows you to test and debug your workloads on Kubernetes locally -before deploying. - -## Create a Kubernetes YAML file - -In your `docker-dotnet-sample` directory, create a file named -`docker-dotnet-kubernetes.yaml`. Open the file in an IDE or text editor and add -the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker -username and the name of the repository that you created in [Configure CI/CD for -your .NET application](configure-ci-cd.md). - -```yaml -apiVersion: apps/v1 -kind: Deployment -metadata: - labels: - service: server - name: server - namespace: default -spec: - replicas: 1 - selector: - matchLabels: - service: server - strategy: {} - template: - metadata: - labels: - service: server - spec: - initContainers: - - name: wait-for-db - image: busybox:1.28 - command: - [ - "sh", - "-c", - 'until nc -zv db 5432; do echo "waiting for db"; sleep 2; done;', - ] - containers: - - image: DOCKER_USERNAME/REPO_NAME - name: server - imagePullPolicy: Always - ports: - - containerPort: 8080 - hostPort: 8080 - protocol: TCP - resources: {} - restartPolicy: Always -status: {} ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - labels: - service: db - name: db - namespace: default -spec: - replicas: 1 - selector: - matchLabels: - service: db - strategy: - type: Recreate - template: - metadata: - labels: - service: db - spec: - containers: - - env: - - name: POSTGRES_DB - value: example - - name: POSTGRES_PASSWORD - value: example - image: postgres:18 - name: db - ports: - - containerPort: 5432 - protocol: TCP - resources: {} - restartPolicy: Always -status: {} ---- -apiVersion: v1 -kind: Service -metadata: - labels: - service: server - name: server - namespace: default -spec: - type: NodePort - ports: - - name: "8080" - port: 8080 - targetPort: 8080 - nodePort: 30001 - selector: - service: server -status: - loadBalancer: {} ---- -apiVersion: v1 -kind: Service -metadata: - labels: - service: db - name: db - namespace: default -spec: - ports: - - name: "5432" - port: 5432 - targetPort: 5432 - selector: - service: db -status: - loadBalancer: {} -``` - -In this Kubernetes YAML file, there are four objects, separated by the `---`. In addition to a Service and Deployment for the database, the other two objects are: - -- A Deployment, describing a scalable group of identical pods. In this case, - you'll get just one replica, or copy of your pod. That pod, which is - described under `template`, has just one container in it. The container is - created from the image built by GitHub Actions in [Configure CI/CD for your - .NET application](configure-ci-cd.md). -- A NodePort service, which will route traffic from port 30001 on your host to - port 8080 inside the pods it routes to, allowing you to reach your app - from the network. - -To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). - -## Deploy and check your application - -1. In a terminal, navigate to the `docker-dotnet-sample` directory - and deploy your application to Kubernetes. - - ```console - $ kubectl apply -f docker-dotnet-kubernetes.yaml - ``` - - You should see output that looks like the following, indicating your Kubernetes objects were created successfully. - - ```shell - deployment.apps/db created - service/db created - deployment.apps/server created - service/server created - ``` - -2. Make sure everything worked by listing your deployments. - - ```console - $ kubectl get deployments - ``` - - Your deployment should be listed as follows: - - ```shell - NAME READY UP-TO-DATE AVAILABLE AGE - db 1/1 1 1 76s - server 1/1 1 1 76s - ``` - - This indicates all of the pods are up and running. Do the same check for your services. - - ```console - $ kubectl get services - ``` - - You should get output like the following. - - ```shell - NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE - db ClusterIP 10.96.156.90 5432/TCP 2m8s - kubernetes ClusterIP 10.96.0.1 443/TCP 164m - server NodePort 10.102.94.225 8080:30001/TCP 2m8s - ``` - - In addition to the default `kubernetes` service, you can see your `server` service and `db` service. The `server` service is accepting traffic on port 30001/TCP. - -3. Open a browser and visit your app at `localhost:30001`. You should see your - application. - -4. Run the following command to tear down your application. - - ```console - $ kubectl delete -f docker-dotnet-kubernetes.yaml - ``` - -## Summary - -In this section, you learned how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. - -Related information: - -- [Kubernetes documentation](https://kubernetes.io/docs/home/) -- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) -- [Swarm mode overview](/manuals/engine/swarm/_index.md) diff --git a/content/guides/dotnet/develop.md b/content/guides/dotnet/develop.md deleted file mode 100644 index b07c522497c8..000000000000 --- a/content/guides/dotnet/develop.md +++ /dev/null @@ -1,425 +0,0 @@ ---- -title: Use containers for .NET development -linkTitle: Develop your app -weight: 20 -keywords: .net, development -description: Learn how to develop your .NET application locally using containers. -aliases: - - /language/dotnet/develop/ - - /guides/language/dotnet/develop/ ---- - -## Prerequisites - -Complete [Containerize a .NET application](containerize.md). - -## Overview - -In this section, you'll learn how to set up a development environment for your containerized application. This includes: - -- Adding a local database and persisting data -- Configuring Compose to automatically update your running Compose services as you edit and save your code -- Creating a development container that contains the .NET Core SDK tools and dependencies - -## Update the application - -This section uses a different branch of the `docker-dotnet-sample` repository -that contains an updated .NET application. The updated application is on the -`add-db` branch of the repository you cloned in [Containerize a .NET -application](containerize.md). - -To get the updated code, you need to checkout the `add-db` branch. For the changes you made in [Containerize a .NET application](containerize.md), for this section, you can stash them. In a terminal, run the following commands in the `docker-dotnet-sample` directory. - -1. Stash any previous changes. - - ```console - $ git stash -u - ``` - -2. Check out the new branch with the updated application. - - ```console - $ git checkout add-db - ``` - -In the `add-db` branch, only the .NET application has been updated. None of the Docker assets have been updated yet. - -You should now have the following in your `docker-dotnet-sample` directory. - -```text -├── docker-dotnet-sample/ -│ ├── .git/ -│ ├── src/ -│ │ ├── Data/ -│ │ ├── Models/ -│ │ ├── Pages/ -│ │ ├── Properties/ -│ │ ├── wwwroot/ -│ │ ├── appsettings.Development.json -│ │ ├── appsettings.json -│ │ ├── myWebApp.csproj -│ │ └── Program.cs -│ ├── tests/ -│ │ ├── tests.csproj -│ │ ├── UnitTest1.cs -│ │ └── Usings.cs -│ ├── .dockerignore -│ ├── .gitignore -│ ├── compose.yaml -│ ├── Dockerfile -│ └── README.md -``` - -## Add a local database and persist data - -You can use containers to set up local services, like a database. In this section, you'll update the `compose.yaml` file to define a database service and a volume to persist data. - -Open the `compose.yaml` file in an IDE or text editor. You'll notice it -already contains commented-out instructions for a PostgreSQL database and volume. - -Open `docker-dotnet-sample/src/appsettings.json` in an IDE or text editor. You'll -notice the connection string with all the database information. The -`compose.yaml` already contains this information, but it's commented out. -Uncomment the database instructions in the `compose.yaml` file. - -The following is the updated `compose.yaml` file. - -```yaml {hl_lines="8-33"} -services: - server: - build: - context: . - target: final - ports: - - 8080:8080 - depends_on: - db: - condition: service_healthy - db: - image: postgres:18 - restart: always - user: postgres - secrets: - - db-password - volumes: - - db-data:/var/lib/postgresql - environment: - - POSTGRES_DB=example - - POSTGRES_PASSWORD_FILE=/run/secrets/db-password - expose: - - 5432 - healthcheck: - test: ["CMD", "pg_isready"] - interval: 10s - timeout: 5s - retries: 5 -volumes: - db-data: -secrets: - db-password: - file: db/password.txt -``` - -> [!NOTE] -> -> To learn more about the instructions in the Compose file, see [Compose file -> reference](/reference/compose-file/). - -Before you run the application using Compose, notice that this Compose file uses -`secrets` and specifies a `password.txt` file to hold the database's password. -You must create this file as it's not included in the source repository. - -In the `docker-dotnet-sample` directory, create a new directory named `db` and -inside that directory create a file named `password.txt`. Open `password.txt` in an IDE or text editor and add the following password. The password must be on a single line, with no additional lines in the file. - -```text -example -``` - -Save and close the `password.txt` file. - -You should now have the following in your `docker-dotnet-sample` directory. - -```text -├── docker-dotnet-sample/ -│ ├── .git/ -│ ├── db/ -│ │ └── password.txt -│ ├── src/ -│ ├── tests/ -│ ├── .dockerignore -│ ├── .gitignore -│ ├── compose.yaml -│ ├── Dockerfile -│ └── README.md -``` - -Run the following command to start your application. - -```console -$ docker compose up --build -``` - -Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see a simple web application with the text `Student name is`. - -The application doesn't display a name because the database is empty. For this application, you need to access the database and then add records. - -## Add records to the database - -For the sample application, you must access the database directly to create sample records. - -You can run commands inside the database container using the `docker exec` -command. Before running that command, you must get the ID of the database -container. Open a new terminal window and run the following command to list all -your running containers. - -```console -$ docker container ls -``` - -You should see output like the following. - -```console -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -cb36e310aa7e docker-dotnet-sample-server "dotnet myWebApp.dll" About a minute ago Up About a minute 0.0.0.0:8080->8080/tcp docker-dotnet-sample-server-1 -39fdcf0aff7b postgres:18 "docker-entrypoint.s…" About a minute ago Up About a minute (healthy) 5432/tcp docker-dotnet-sample-db-1 -``` - -In the previous example, the container ID is `39fdcf0aff7b`. Run the following command to connect to the postgres database in the container. Replace the container ID with your own container ID. - -```console -$ docker exec -it 39fdcf0aff7b psql -d example -U postgres -``` - -And finally, insert a record into the database. - -```console -example=# INSERT INTO "Students" ("ID", "LastName", "FirstMidName", "EnrollmentDate") VALUES (DEFAULT, 'Whale', 'Moby', '2013-03-20'); -``` - -You should see output like the following. - -```console -INSERT 0 1 -``` - -Close the database connection and exit the container shell by running `exit`. - -```console -example=# exit -``` - -## Verify that data persists in the database - -Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see a simple web application with the text `Student name is Moby Whale`. - -Press `ctrl+c` in the terminal to stop your application. - -In the terminal, run `docker compose rm` to remove your containers and then run `docker compose up` to run your application again. - -```console -$ docker compose rm -$ docker compose up --build -``` - -Refresh [http://localhost:8080](http://localhost:8080) in your browser and verify that the student name persisted, even after the containers were removed and ran again. - -Press `ctrl+c` in the terminal to stop your application. - -## Automatically update services - -Use Compose Watch to automatically update your running Compose services as you edit and save your code. For more details about Compose Watch, see [Use Compose Watch](/manuals/compose/how-tos/file-watch.md). - -Open your `compose.yaml` file in an IDE or text editor and then add the Compose Watch instructions. The following is the updated `compose.yaml` file. - -```yaml {hl_lines="11-14"} -services: - server: - build: - context: . - target: final - ports: - - 8080:8080 - depends_on: - db: - condition: service_healthy - develop: - watch: - - action: rebuild - path: . - db: - image: postgres:18 - restart: always - user: postgres - secrets: - - db-password - volumes: - - db-data:/var/lib/postgresql - environment: - - POSTGRES_DB=example - - POSTGRES_PASSWORD_FILE=/run/secrets/db-password - expose: - - 5432 - healthcheck: - test: ["CMD", "pg_isready"] - interval: 10s - timeout: 5s - retries: 5 -volumes: - db-data: -secrets: - db-password: - file: db/password.txt -``` - -Run the following command to run your application with Compose Watch. - -```console -$ docker compose watch -``` - -Open a browser and verify that the application is running at [http://localhost:8080](http://localhost:8080). - -Any changes to the application's source files on your local machine will now be -immediately reflected in the running container. - -Open `docker-dotnet-sample/src/Pages/Index.cshtml` in an IDE or text editor and update the student name text on line 13 from `Student name is` to `Student name:`. - -```diff --

Student name is @Model.StudentName

-+

Student name: @Model.StudentName

-``` - -Save the changes to `Index.cshtml` and then wait a few seconds for the application to rebuild. Refresh [http://localhost:8080](http://localhost:8080) in your browser and verify that the updated text appears. - -Press `ctrl+c` in the terminal to stop your application. - -## Create a development container - -At this point, when you run your containerized application, it's using the .NET runtime image. While this small image is good for production, it lacks the SDK tools and dependencies you may need when developing. Also, during development, you may not need to run `dotnet publish`. You can use multi-stage builds to build stages for both development and production in the same Dockerfile. For more details, see [Multi-stage builds](/manuals/build/building/multi-stage.md). - -Add a new development stage to your Dockerfile and update your `compose.yaml` file to use this stage for local development. - -The following is the updated Dockerfile. - -{{< tabs >}} -{{< tab name="Using Docker Hardened Images" >}} - -```Dockerfile {hl_lines="10-13"} -# syntax=docker/dockerfile:1 - -FROM --platform=$BUILDPLATFORM dhi.io/dotnet:10-sdk AS build -ARG TARGETARCH -COPY . /source -WORKDIR /source/src -RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages \ - dotnet publish -a ${TARGETARCH/amd64/x64} --use-current-runtime --self-contained false -o /app - -FROM dhi.io/dotnet:10-sdk AS development -COPY . /source -WORKDIR /source/src -CMD dotnet run --no-launch-profile - -FROM dhi.io/aspnetcore:10 AS final -WORKDIR /app -COPY --from=build /app . -ENTRYPOINT ["dotnet", "myWebApp.dll"] -``` - -{{< /tab >}} -{{< tab name="Using the official .NET 10 image" >}} - -```Dockerfile {hl_lines="10-13"} -# syntax=docker/dockerfile:1 - -FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:10.0-alpine AS build -ARG TARGETARCH -COPY . /source -WORKDIR /source/src -RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages \ - dotnet publish -a ${TARGETARCH/amd64/x64} --use-current-runtime --self-contained false -o /app - -FROM mcr.microsoft.com/dotnet/sdk:10.0-alpine AS development -COPY . /source -WORKDIR /source/src -CMD dotnet run --no-launch-profile - -FROM mcr.microsoft.com/dotnet/aspnet:10.0-alpine AS final -WORKDIR /app -COPY --from=build /app . -ARG UID=10001 -RUN adduser \ - --disabled-password \ - --gecos "" \ - --home "/nonexistent" \ - --shell "/sbin/nologin" \ - --no-create-home \ - --uid "${UID}" \ - appuser -USER appuser -ENTRYPOINT ["dotnet", "myWebApp.dll"] -``` - -{{< /tab >}} -{{< /tabs >}} - -The following is the updated `compose.yaml` file. - -```yaml {hl_lines=[5,15,16]} -services: - server: - build: - context: . - target: development - ports: - - 8080:8080 - depends_on: - db: - condition: service_healthy - develop: - watch: - - action: rebuild - path: . - environment: - - ASPNETCORE_ENVIRONMENT=Development - db: - image: postgres:18 - restart: always - user: postgres - secrets: - - db-password - volumes: - - db-data:/var/lib/postgresql - environment: - - POSTGRES_DB=example - - POSTGRES_PASSWORD_FILE=/run/secrets/db-password - expose: - - 5432 - healthcheck: - test: ["CMD", "pg_isready"] - interval: 10s - timeout: 5s - retries: 5 -volumes: - db-data: -secrets: - db-password: - file: db/password.txt -``` - -Your containerized application will now use the SDK image (either `dhi.io/dotnet:10-sdk` for DHI or `mcr.microsoft.com/dotnet/sdk:10.0-alpine` for official images), which includes development tools like `dotnet test`. Continue to the next section to learn how you can run `dotnet test`. - -## Summary - -In this section, you took a look at setting up your Compose file to add a local -database and persist data. You also learned how to use Compose Watch to automatically rebuild and run your container when you update your code. And finally, you learned how to create a development container that contains the SDK tools and dependencies needed for development. - -Related information: - -- [Compose file reference](/reference/compose-file/) -- [Compose file watch](/manuals/compose/how-tos/file-watch.md) -- [Multi-stage builds](/manuals/build/building/multi-stage.md) - -## Next steps - -In the next section, you'll learn how to run unit tests using Docker. diff --git a/content/guides/dotnet/run-tests.md b/content/guides/dotnet/run-tests.md deleted file mode 100644 index 196a7b6311c6..000000000000 --- a/content/guides/dotnet/run-tests.md +++ /dev/null @@ -1,149 +0,0 @@ ---- -title: Run .NET tests in a container -linkTitle: Run your tests -weight: 30 -keywords: .NET, test -description: Learn how to run your .NET tests in a container. -aliases: - - /language/dotnet/run-tests/ - - /guides/language/dotnet/run-tests/ ---- - -## Prerequisites - -Complete all the previous sections of this guide, starting with [Containerize a .NET application](containerize.md). - -## Overview - -Testing is an essential part of modern software development. Testing can mean a -lot of things to different development teams. There are unit tests, integration -tests and end-to-end testing. In this guide you take a look at running your unit -tests in Docker when developing and when building. - -## Run tests when developing locally - -The sample application already has an xUnit test inside the `tests` directory. When developing locally, you can use Compose to run your tests. - -Run the following command in the `docker-dotnet-sample` directory to run the tests inside a container. - -```console -$ docker compose run --build --rm server dotnet test /source/tests -``` - -You should see output that contains the following. - -```console -Starting test execution, please wait... -A total of 1 test files matched the specified pattern. - -Passed! - Failed: 0, Passed: 1, Skipped: 0, Total: 1, Duration: < 1 ms - /source/tests/bin/Debug/net10.0/tests.dll (net10.0) -``` - -To learn more about the command, see [docker compose run](/reference/cli/docker/compose/run/). - -## Run tests when building - -To run your tests when building, you need to update your Dockerfile. You can create a new test stage that runs the tests, or run the tests in the existing build stage. For this guide, update the Dockerfile to run the tests in the build stage. - -The following is the updated Dockerfile. - -{{< tabs >}} -{{< tab name="Using Docker Hardened Images" >}} - -```dockerfile {hl_lines="9"} -# syntax=docker/dockerfile:1 - -FROM --platform=$BUILDPLATFORM dhi.io/dotnet:10-sdk AS build -ARG TARGETARCH -COPY . /source -WORKDIR /source/src -RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages \ - dotnet publish -a ${TARGETARCH/amd64/x64} --use-current-runtime --self-contained false -o /app -RUN dotnet test /source/tests - -FROM dhi.io/dotnet:10-sdk AS development -COPY . /source -WORKDIR /source/src -CMD dotnet run --no-launch-profile - -FROM dhi.io/aspnetcore:10 AS final -WORKDIR /app -COPY --from=build /app . -ENTRYPOINT ["dotnet", "myWebApp.dll"] -``` - -{{< /tab >}} -{{< tab name="Using the official .NET 10 image" >}} - -```dockerfile {hl_lines="9"} -# syntax=docker/dockerfile:1 - -FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:10.0-alpine AS build -ARG TARGETARCH -COPY . /source -WORKDIR /source/src -RUN --mount=type=cache,id=nuget,target=/root/.nuget/packages \ - dotnet publish -a ${TARGETARCH/amd64/x64} --use-current-runtime --self-contained false -o /app -RUN dotnet test /source/tests - -FROM mcr.microsoft.com/dotnet/sdk:10.0-alpine AS development -COPY . /source -WORKDIR /source/src -CMD dotnet run --no-launch-profile - -FROM mcr.microsoft.com/dotnet/aspnet:10.0-alpine AS final -WORKDIR /app -COPY --from=build /app . -ARG UID=10001 -RUN adduser \ - --disabled-password \ - --gecos "" \ - --home "/nonexistent" \ - --shell "/sbin/nologin" \ - --no-create-home \ - --uid "${UID}" \ - appuser -USER appuser -ENTRYPOINT ["dotnet", "myWebApp.dll"] -``` - -{{< /tab >}} -{{< /tabs >}} - -Run the following command to build an image using the build stage as the target and view the test results. Include `--progress=plain` to view the build output, `--no-cache` to ensure the tests always run, and `--target build` to target the build stage. - -```console -$ docker build -t dotnet-docker-image-test --progress=plain --no-cache --target build . -``` - -You should see output containing the following. - -```console -#11 [build 5/5] RUN dotnet test /source/tests -#11 1.564 Determining projects to restore... -#11 3.421 Restored /source/src/myWebApp.csproj (in 1.02 sec). -#11 19.42 Restored /source/tests/tests.csproj (in 17.05 sec). -#11 27.91 myWebApp -> /source/src/bin/Debug/net10.0/myWebApp.dll -#11 28.47 tests -> /source/tests/bin/Debug/net10.0/tests.dll -#11 28.49 Test run for /source/tests/bin/Debug/net10.0/tests.dll (.NETCoreApp,Version=v10.0) -#11 28.67 Microsoft (R) Test Execution Command Line Tool Version 17.3.3 (x64) -#11 28.67 Copyright (c) Microsoft Corporation. All rights reserved. -#11 28.68 -#11 28.97 Starting test execution, please wait... -#11 29.03 A total of 1 test files matched the specified pattern. -#11 32.07 -#11 32.08 Passed! - Failed: 0, Passed: 1, Skipped: 0, Total: 1, Duration: < 1 ms - /source/tests/bin/Debug/net10.0/tests.dll (net10.0) -#11 DONE 32.2s -``` - -## Summary - -In this section, you learned how to run tests when developing locally using Compose and how to run tests when building your image. - -Related information: - -- [docker compose run](/reference/cli/docker/compose/run/) - -## Next steps - -Next, you’ll learn how to set up a CI/CD pipeline using GitHub Actions. diff --git a/content/guides/frameworks/laravel/_index.md b/content/guides/frameworks/laravel/_index.md index 4363ca51cd52..aaa7846c9524 100644 --- a/content/guides/frameworks/laravel/_index.md +++ b/content/guides/frameworks/laravel/_index.md @@ -4,23 +4,19 @@ linkTitle: Laravel summary: Learn how to efficiently set up Laravel development and production environments using Docker Compose. description: A guide on using Docker Compose to manage Laravel applications for development and production, covering container configurations and service management. keywords: laravel, php, docker compose, web framework, development, production -tags: [frameworks] -languages: [php] aliases: - /frameworks/laravel/ + - /guides/frameworks/laravel/common-questions/ + - /guides/frameworks/laravel/development-setup/ + - /guides/frameworks/laravel/prerequisites/ + - /guides/frameworks/laravel/production-setup/ params: + tags: [cicd] time: 30 minutes - resource_links: - - title: Laravel - url: https://laravel.com/ - - title: Docker Compose - url: /compose/ - - title: Use Compose in production - url: /compose/how-tos/production/ - - title: Repository with examples url: https://github.com/dockersamples/laravel-docker-examples --- + Laravel is a popular PHP framework that allows developers to build web applications quickly and effectively. Docker Compose simplifies the management of development and production environments by defining essential services, like PHP, a web server, and a database, in a single YAML file. This guide provides a streamlined approach to setting up a robust Laravel environment using Docker Compose, focusing on simplicity and efficiency. > **Acknowledgment** @@ -45,3 +41,823 @@ This guide is intended for educational purposes, helping developers adapt and op - Developers who work with Laravel and want to streamline environment management. - DevOps engineers seeking efficient ways to manage and deploy Laravel applications. + +## Prerequisites for Setting Up Laravel with Docker Compose + +Before you begin setting up Laravel with Docker Compose, make sure you meet the following prerequisites: + +### Docker and Docker Compose + +You need Docker and Docker Compose installed on your system. Docker allows you to containerize applications, and Docker Compose helps you manage multi-container applications. + +- Docker: Make sure Docker is installed and running on your machine. Refer to the [Docker installation guide](/get-docker/) to install Docker. +- Docker Compose: Docker Compose is included with Docker Desktop, but you can also follow the [Docker Compose installation guide](/compose/install/) if needed. + +### Basic understanding of Docker and containers + +A fundamental understanding of Docker and how containers work will be helpful. If you're new to Docker, consider reviewing the [Docker Overview](/get-started/overview/) to familiarize yourself with containerization concepts. + +### Basic knowledge of Laravel + +This guide assumes you have a basic understanding of Laravel and PHP. Familiarity with Laravel’s command-line tools, such as [Artisan](https://laravel.com/docs/12.x/artisan), and its project structure is important for following the instructions. + +- Laravel CLI: You should be comfortable using Laravel’s command-line tool (`artisan`). +- Laravel Project Structure: Familiarize yourself with Laravel’s folder structure (`app`, `config`, `routes`, `tests`, etc.). + +## Laravel Production Setup with Docker Compose + +This guide demonstrates how to set up a production-ready Laravel environment using Docker and Docker Compose. This configuration is designed for streamlined, scalable, and secure Laravel application deployments. + +> [!NOTE] +> To experiment with a ready-to-run configuration, download the [Laravel Docker Examples](https://github.com/dockersamples/laravel-docker-examples) repository. It contains pre-configured setups for both development and production. + +### Project structure + +```plaintext +my-laravel-app/ +├── app/ +├── bootstrap/ +├── config/ +├── database/ +├── public/ +├── docker/ +│ ├── common/ +│ │ └── php-fpm/ +│ │ ├── Dockerfile +│ │ └── conf.d/ +│ │ └── 20-status-path.conf +│ ├── development/ +│ ├── production/ +│ │ ├── php-fpm/ +│ │ │ └── entrypoint.sh +│ │ └── nginx +│ │ ├── Dockerfile +│ │ └── nginx.conf +├── compose.dev.yaml +├── compose.prod.yaml +├── .dockerignore +├── .env +├── vendor/ +├── ... +``` + +This layout represents a typical Laravel project, with Docker configurations stored in a unified `docker` directory. You’ll find **two** Compose files — `compose.dev.yaml` (for development) and `compose.prod.yaml` (for production) — to keep your environments separate and manageable. + +### Create a Dockerfile for PHP-FPM (production) + +For production, the `php-fpm` Dockerfile creates an optimized image with only the PHP extensions and libraries your application needs. As demonstrated in the [GitHub example](https://github.com/dockersamples/laravel-docker-examples), a single Dockerfile with multi-stage builds maintains consistency and reduces duplication between development and production. The following snippet shows only the production-related stages: + +```dockerfile +# Stage 1: Build environment and Composer dependencies +FROM php:8.5-fpm AS builder + +# Install system dependencies and PHP extensions for Laravel with MySQL/PostgreSQL support. +# Dependencies in this stage are only required for building the final image. +# Node.js and asset building are handled in the Nginx stage, not here. +RUN apt-get update && apt-get install -y --no-install-recommends \ + curl \ + unzip \ + libpq-dev \ + libonig-dev \ + libssl-dev \ + libxml2-dev \ + libcurl4-openssl-dev \ + libicu-dev \ + libzip-dev \ + && docker-php-ext-install -j$(nproc) \ + pdo_mysql \ + pdo_pgsql \ + pgsql \ + intl \ + zip \ + bcmath \ + soap \ + && pecl install redis \ + && docker-php-ext-enable redis \ + && apt-get autoremove -y && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* + +# Set the working directory inside the container +WORKDIR /var/www + +# Copy the entire Laravel application code into the container +# ----------------------------------------------------------- +# In Laravel, `composer install` may trigger scripts +# needing access to application code. +# For example, the `post-autoload-dump` event might execute +# Artisan commands like `php artisan package:discover`. If the +# application code (including the `artisan` file) is not +# present, these commands will fail, leading to build errors. +# +# By copying the entire application code before running +# `composer install`, we ensure that all necessary files are +# available, allowing these scripts to run successfully. +# In other cases, it would be possible to copy composer files +# first, to leverage Docker's layer caching mechanism. +# ----------------------------------------------------------- +COPY . /var/www + +# Install Composer and dependencies +RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer \ + && composer install --no-dev --optimize-autoloader --no-interaction --no-progress --prefer-dist + +# Stage 2: Production environment +FROM php:8.5-fpm AS production + +# Install only runtime libraries needed in production +# libfcgi-bin and procps are required for the php-fpm-healthcheck script +RUN apt-get update && apt-get install -y --no-install-recommends \ + libpq-dev \ + libicu-dev \ + libzip-dev \ + libfcgi-bin \ + procps \ + && apt-get autoremove -y && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* + +# Download and install php-fpm health check script +RUN curl -o /usr/local/bin/php-fpm-healthcheck \ + https://raw.githubusercontent.com/renatomefi/php-fpm-healthcheck/master/php-fpm-healthcheck \ + && chmod +x /usr/local/bin/php-fpm-healthcheck + +# Copy the initialization script +COPY ./docker/php-fpm/entrypoint.sh /usr/local/bin/entrypoint.sh +RUN chmod +x /usr/local/bin/entrypoint.sh + +# Copy the initial storage structure +COPY ./storage /var/www/storage-init + +# Copy PHP extensions and libraries from the builder stage +COPY --from=builder /usr/local/lib/php/extensions/ /usr/local/lib/php/extensions/ +COPY --from=builder /usr/local/etc/php/conf.d/ /usr/local/etc/php/conf.d/ +COPY --from=builder /usr/local/bin/docker-php-ext-* /usr/local/bin/ + +# Use the recommended production PHP configuration +# ----------------------------------------------------------- +# PHP provides development and production configurations. +# Here, we replace the default php.ini with the production +# version to apply settings optimized for performance and +# security in a live environment. +# ----------------------------------------------------------- +RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini" + +# Keep the image-provided FPM global config intact and add pool overrides separately +COPY ./docker/common/php-fpm/conf.d/*.conf /usr/local/etc/php-fpm.d/ +# Update the variables_order to include E (for ENV) +#RUN sed -i 's/variables_order = "GPCS"/variables_order = "EGPCS"/' "$PHP_INI_DIR/php.ini" + +# Copy the application code and dependencies from the build stage +COPY --from=builder /var/www /var/www + +# Set working directory +WORKDIR /var/www + +# Ensure correct permissions +RUN chown -R www-data:www-data /var/www + +# Switch to the non-privileged user to run the application +USER www-data + +# Change the default command to run the entrypoint script +ENTRYPOINT ["/usr/local/bin/entrypoint.sh"] + +# Expose port 9000 and start php-fpm server +EXPOSE 9000 +CMD ["php-fpm"] +``` + +### Create a Dockerfile for PHP-CLI (production) + +For production, you often need a separate container to run Artisan commands, migrations, and other CLI tasks. In most cases you can run these commands by reusing existing PHP-FPM container: + +```console +$ docker compose -f compose.prod.yaml exec php-fpm php artisan route:list +``` + +If you need a separate CLI container with different extensions or strict separation of concerns, consider a php-cli Dockerfile: + +```dockerfile +# Stage 1: Build environment and Composer dependencies +FROM php:8.5-cli AS builder + +# Install system dependencies and PHP extensions required for Laravel + MySQL/PostgreSQL support +# Some dependencies are required for PHP extensions only in the build stage +RUN apt-get update && apt-get install -y --no-install-recommends \ + curl \ + unzip \ + libpq-dev \ + libonig-dev \ + libssl-dev \ + libxml2-dev \ + libcurl4-openssl-dev \ + libicu-dev \ + libzip-dev \ + && docker-php-ext-install -j$(nproc) \ + pdo_mysql \ + pdo_pgsql \ + pgsql \ + intl \ + zip \ + bcmath \ + soap \ + && pecl install redis \ + && docker-php-ext-enable redis \ + && apt-get autoremove -y && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* + +# Set the working directory inside the container +WORKDIR /var/www + +# Copy the entire Laravel application code into the container +COPY . /var/www + +# Install Composer and dependencies +RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer \ + && composer install --no-dev --optimize-autoloader --no-interaction --no-progress --prefer-dist + +# Stage 2: Production environment +FROM php:8.5-cli + +# Install client libraries required for php extensions in runtime +RUN apt-get update && apt-get install -y --no-install-recommends \ + libpq-dev \ + libicu-dev \ + libzip-dev \ + && apt-get autoremove -y && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* + +# Copy PHP extensions and libraries from the builder stage +COPY --from=builder /usr/local/lib/php/extensions/ /usr/local/lib/php/extensions/ +COPY --from=builder /usr/local/etc/php/conf.d/ /usr/local/etc/php/conf.d/ +COPY --from=builder /usr/local/bin/docker-php-ext-* /usr/local/bin/ + +# Use the default production configuration for PHP runtime arguments +RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini" + +# Copy the application code and dependencies from the build stage +COPY --from=builder /var/www /var/www + +# Set working directory +WORKDIR /var/www + +# Ensure correct permissions +RUN chown -R www-data:www-data /var/www + +# Switch to the non-privileged user to run the application +USER www-data + +# Default command: Provide a bash shell to allow running any command +CMD ["bash"] +``` + +This Dockerfile is similar to the PHP-FPM Dockerfile, but it uses the `php:8.5-cli` image as the base image and sets up the container for running CLI commands. + +### Create a Dockerfile for Nginx (production) + +Nginx serves as the web server for the Laravel application. You can include static assets directly to the container. Here's an example of possible Dockerfile for Nginx: + +```dockerfile +# docker/nginx/Dockerfile +# Stage 1: Build assets +FROM debian AS builder + +# Install Node.js and build tools +RUN apt-get update && apt-get install -y --no-install-recommends \ + curl \ + nodejs \ + npm \ + && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* + +# Set working directory +WORKDIR /var/www + +# Copy Laravel application code +COPY . /var/www + +# Install Node.js dependencies and build assets +RUN npm install && npm run build + +# Stage 2: Nginx production image +FROM nginx:alpine + +# Copy custom Nginx configuration +# ----------------------------------------------------------- +# Replace the default Nginx configuration with our custom one +# that is optimized for serving a Laravel application. +# ----------------------------------------------------------- +COPY ./docker/nginx/nginx.conf /etc/nginx/nginx.conf + +# Copy Laravel's public assets from the builder stage +# ----------------------------------------------------------- +# We only need the 'public' directory from our Laravel app. +# ----------------------------------------------------------- +COPY --from=builder /var/www/public /var/www/public + +# Set the working directory to the public folder +WORKDIR /var/www/public + +# Expose port 80 and start Nginx +EXPOSE 80 +CMD ["nginx", "-g", "daemon off;"] +``` + +This Dockerfile uses a multi-stage build to separate the asset building process from the final production image. The first stage installs Node.js and builds the assets, while the second stage sets up the Nginx production image with the optimized configuration and the built assets. + +### Create a Docker Compose configuration for production + +To bring all the services together, create a `compose.prod.yaml` file that defines the services, volumes, and networks for the production environment. Here's an example configuration: + +```yaml +services: + web: + build: + context: . + dockerfile: ./docker/production/nginx/Dockerfile + restart: unless-stopped # Automatically restart unless the service is explicitly stopped + volumes: + # Mount the 'laravel-storage' volume to '/var/www/storage' inside the container. + # ----------------------------------------------------------- + # This volume stores persistent data like uploaded files and cache. + # The ':ro' option mounts it as read-only in the 'web' service because Nginx only needs to read these files. + # The 'php-fpm' service mounts the same volume without ':ro' to allow write operations. + # ----------------------------------------------------------- + - laravel-storage-production:/var/www/storage:ro + networks: + - laravel-production + ports: + # Map port 80 inside the container to the port specified by 'NGINX_PORT' on the host machine. + # ----------------------------------------------------------- + # This allows external access to the Nginx web server running inside the container. + # For example, if 'NGINX_PORT' is set to '8080', accessing 'http://localhost:8080' will reach the application. + # ----------------------------------------------------------- + - "${NGINX_PORT:-80}:80" + depends_on: + php-fpm: + condition: service_healthy # Wait for php-fpm health check + + php-fpm: + # For the php-fpm service, we will create a custom image to install the necessary PHP extensions and setup proper permissions. + build: + context: . + dockerfile: ./docker/common/php-fpm/Dockerfile + target: production # Use the 'production' stage in the Dockerfile + restart: unless-stopped + volumes: + - laravel-storage-production:/var/www/storage # Mount the storage volume + env_file: + - .env + networks: + - laravel-production + healthcheck: + test: ["CMD-SHELL", "php-fpm-healthcheck || exit 1"] + interval: 10s + timeout: 5s + retries: 3 + # The 'depends_on' attribute with 'condition: service_healthy' ensures that + # this service will not start until the 'postgres' service passes its health check. + # This prevents the application from trying to connect to the database before it's ready. + depends_on: + postgres: + condition: service_healthy + + # The 'php-cli' service provides a command-line interface for running Artisan commands and other CLI tasks. + # ----------------------------------------------------------- + # This is useful for running migrations, seeders, or any custom scripts. + # It shares the same codebase and environment as the 'php-fpm' service. + # ----------------------------------------------------------- + php-cli: + build: + context: . + dockerfile: ./docker/php-cli/Dockerfile + tty: true # Enables an interactive terminal + stdin_open: true # Keeps standard input open for 'docker exec' + env_file: + - .env + networks: + - laravel-production + + postgres: + image: postgres:18 + restart: unless-stopped + user: postgres + ports: + - "${POSTGRES_PORT}:5432" + environment: + - POSTGRES_DB=${POSTGRES_DATABASE} + - POSTGRES_USER=${POSTGRES_USERNAME} + - POSTGRES_PASSWORD=${POSTGRES_PASSWORD} + volumes: + - postgres-data-production:/var/lib/postgresql + networks: + - laravel-production + # Health check for PostgreSQL + # ----------------------------------------------------------- + # Health checks allow Docker to determine if a service is operational. + # The 'pg_isready' command checks if PostgreSQL is ready to accept connections. + # This prevents dependent services from starting before the database is ready. + # ----------------------------------------------------------- + healthcheck: + test: ["CMD", "pg_isready"] + interval: 10s + timeout: 5s + retries: 5 + + redis: + image: redis:alpine + restart: unless-stopped # Automatically restart unless the service is explicitly stopped + networks: + - laravel-production + # Health check for Redis + # ----------------------------------------------------------- + # Checks if Redis is responding to the 'PING' command. + # This ensures that the service is not only running but also operational. + # ----------------------------------------------------------- + healthcheck: + test: ["CMD", "redis-cli", "ping"] + interval: 10s + timeout: 5s + retries: 3 + +networks: + # Attach the service to the 'laravel-production' network. + # ----------------------------------------------------------- + # This custom network allows all services within it to communicate using their service names as hostnames. + # For example, 'php-fpm' can connect to 'postgres' by using 'postgres' as the hostname. + # ----------------------------------------------------------- + laravel-production: + +volumes: + postgres-data-production: + laravel-storage-production: +``` + +> [!NOTE] +> Ensure you have an `.env` file at the root of your Laravel project with the necessary configurations to match the Docker Compose setup. + +### Running your production environment + +To start the production environment, run: + +```console +$ docker compose -f compose.prod.yaml up --build -d +``` + +This command will build and start all the services in detached mode, providing a scalable and production-ready setup for your Laravel application. + +### Summary + +By setting up a Docker Compose environment for Laravel in production, you ensure that your application is optimized for performance, scalable, and secure. This setup makes deployments consistent and easier to manage, reducing the likelihood of errors due to differences between environments. + +## Laravel Development Setup with Docker Compose + +This guide demonstrates how to configure a **development** environment for a Laravel application using Docker and Docker Compose. It builds **on top of** the production image for PHP-FPM and then adds developer-focused features—like Xdebug—to streamline debugging. By basing the development container on a known production image, you keep both environments closely aligned. + +This setup includes PHP-FPM, Nginx, and PostgreSQL services (although you can easily swap PostgreSQL for another database, like MySQL or MariaDB). Everything runs in containers, so you can develop in isolation without altering your host system. + +> [!NOTE] +> To experiment with a ready-to-run configuration, download the [Laravel Docker Examples](https://github.com/dockersamples/laravel-docker-examples) repository. It contains pre-configured setups for both development and production. + +### Project structure + +```plaintext +my-laravel-app/ +├── app/ +├── bootstrap/ +├── config/ +├── database/ +├── public/ +├── docker/ +│ ├── common/ +│ │ └── php-fpm/ +│ │ └── Dockerfile +│ ├── development/ +│ │ ├── php-fpm/ +│ │ │ └── entrypoint.sh +│ │ ├── workspace/ +│ │ │ └── Dockerfile +│ │ └── nginx +│ │ ├── Dockerfile +│ │ └── nginx.conf +│ └── production/ +├── compose.dev.yaml +├── compose.prod.yaml +├── .dockerignore +├── .env +├── vendor/ +├── ... +``` + +This layout represents a typical Laravel project, with Docker configurations stored in a unified `docker` directory. You’ll find **two** Compose files — `compose.dev.yaml` (for development) and `compose.prod.yaml` (for production) — to keep your environments separate and manageable. + +The environment includes a `workspace` service, a sidecar container for tasks like building front-end assets, running Artisan commands, and other CLI tools your project may require. While this extra container may seem unusual, it’s a familiar pattern in solutions like **Laravel Sail** and **Laradock**. It also includes **Xdebug** to aid in debugging. + +### Create a Dockerfile for PHP-FPM + +This Dockerfile **extends** the production image by installing Xdebug and adjusting user permissions to ease local development. That way, your development environment stays consistent with production while still offering extra debug features and improved file mounting. + +```dockerfile +# Builds a dev-only layer on top of the production image +FROM production AS development + +# Use ARGs to define environment variables passed from the Docker build command or Docker Compose. +ARG XDEBUG_ENABLED=true +ARG XDEBUG_MODE=develop,coverage,debug,profile +ARG XDEBUG_HOST=host.docker.internal +ARG XDEBUG_IDE_KEY=DOCKER +ARG XDEBUG_LOG=/dev/stdout +ARG XDEBUG_LOG_LEVEL=0 + +USER root + +# Configure Xdebug if enabled +RUN if [ "${XDEBUG_ENABLED}" = "true" ]; then \ + pecl install xdebug && \ + docker-php-ext-enable xdebug && \ + echo "xdebug.mode=${XDEBUG_MODE}" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini && \ + echo "xdebug.idekey=${XDEBUG_IDE_KEY}" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini && \ + echo "xdebug.log=${XDEBUG_LOG}" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini && \ + echo "xdebug.log_level=${XDEBUG_LOG_LEVEL}" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini && \ + echo "xdebug.client_host=${XDEBUG_HOST}" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini ; \ + echo "xdebug.start_with_request=yes" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini ; \ +fi + +# Add ARGs for syncing permissions +ARG UID=1000 +ARG GID=1000 + +# Create a new user with the specified UID and GID, reusing an existing group if GID exists +RUN if getent group ${GID}; then \ + group_name=$(getent group ${GID} | cut -d: -f1); \ + useradd -m -u ${UID} -g ${GID} -s /bin/bash www; \ + else \ + groupadd -g ${GID} www && \ + useradd -m -u ${UID} -g www -s /bin/bash www; \ + group_name=www; \ + fi + +# Dynamically update php-fpm to use the new user and group +RUN sed -i "s/user = www-data/user = www/g" /usr/local/etc/php-fpm.d/www.conf && \ + sed -i "s/group = www-data/group = $group_name/g" /usr/local/etc/php-fpm.d/www.conf + + +# Set the working directory +WORKDIR /var/www + +# Copy the entrypoint script +COPY ./docker/development/php-fpm/entrypoint.sh /usr/local/bin/entrypoint.sh +RUN chmod +x /usr/local/bin/entrypoint.sh + +# Switch back to the non-privileged user to run the application +USER www-data + +# Change the default command to run the entrypoint script +ENTRYPOINT ["/usr/local/bin/entrypoint.sh"] + +# Expose port 9000 and start php-fpm server +EXPOSE 9000 +CMD ["php-fpm"] +``` + +### Create a Dockerfile for Workspace + +A workspace container provides a dedicated shell for asset compilation, Artisan/Composer commands, and other CLI tasks. This approach follows patterns from Laravel Sail and Laradock, consolidating all development tools into one container for convenience. + +```dockerfile +# docker/development/workspace/Dockerfile +# Use the official PHP CLI image as the base +FROM php:8.5-cli + +# Set environment variables for user and group ID +ARG UID=1000 +ARG GID=1000 +ARG NODE_VERSION=22.0.0 + +# Install system dependencies and build libraries +RUN apt-get update && apt-get install -y --no-install-recommends \ + curl \ + unzip \ + libpq-dev \ + libonig-dev \ + libssl-dev \ + libxml2-dev \ + libcurl4-openssl-dev \ + libicu-dev \ + libzip-dev \ + && docker-php-ext-install -j$(nproc) \ + pdo_mysql \ + pdo_pgsql \ + pgsql \ + intl \ + zip \ + bcmath \ + soap \ + && pecl install redis \ + && docker-php-ext-enable redis \ + && curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer \ + && apt-get autoremove -y && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* + +# Use ARG to define environment variables passed from the Docker build command or Docker Compose. +ARG XDEBUG_ENABLED +ARG XDEBUG_MODE +ARG XDEBUG_HOST +ARG XDEBUG_IDE_KEY +ARG XDEBUG_LOG +ARG XDEBUG_LOG_LEVEL + +# Configure Xdebug if enabled +RUN if [ "${XDEBUG_ENABLED}" = "true" ]; then \ + pecl install xdebug && \ + docker-php-ext-enable xdebug && \ + echo "xdebug.mode=${XDEBUG_MODE}" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini && \ + echo "xdebug.idekey=${XDEBUG_IDE_KEY}" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini && \ + echo "xdebug.log=${XDEBUG_LOG}" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini && \ + echo "xdebug.log_level=${XDEBUG_LOG_LEVEL}" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini && \ + echo "xdebug.client_host=${XDEBUG_HOST}" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini ; \ + echo "xdebug.start_with_request=yes" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini ; \ +fi + +# If the group already exists, use it; otherwise, create the 'www' group +RUN if getent group ${GID}; then \ + useradd -m -u ${UID} -g ${GID} -s /bin/bash www; \ + else \ + groupadd -g ${GID} www && \ + useradd -m -u ${UID} -g www -s /bin/bash www; \ + fi && \ + usermod -aG sudo www && \ + echo 'www ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers + +# Switch to the non-root user to install NVM and Node.js +USER www + +# Install NVM (Node Version Manager) as the www user +RUN export NVM_DIR="$HOME/.nvm" && \ + curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.0/install.sh | bash && \ + [ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh" && \ + nvm install ${NODE_VERSION} && \ + nvm alias default ${NODE_VERSION} && \ + nvm use default + +# Ensure NVM is available for all future shells +RUN echo 'export NVM_DIR="$HOME/.nvm"' >> /home/www/.bashrc && \ + echo '[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"' >> /home/www/.bashrc && \ + echo '[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"' >> /home/www/.bashrc + +# Set the working directory +WORKDIR /var/www + +# Override the entrypoint to avoid the default php entrypoint +ENTRYPOINT [] + +# Default command to keep the container running +CMD ["bash"] +``` + +> [!NOTE] +> If you prefer a **one-service-per-container** approach, simply omit the workspace container and run separate containers for each task. For example, you could use a dedicated `php-cli` container for your PHP scripts, and a `node` container to handle the asset building. + +### Create a Docker Compose configuration for development + +Here's the `compose.yaml` file to set up the development environment: + +```yaml +services: + web: + image: nginx:latest # Using the default Nginx image with custom configuration. + volumes: + # Mount the application code for live updates + - ./:/var/www + # Mount the Nginx configuration file + - ./docker/development/nginx/nginx.conf:/etc/nginx/nginx.conf:ro + ports: + # Map port 80 inside the container to the port specified by 'NGINX_PORT' on the host machine + - "80:80" + environment: + - NGINX_HOST=localhost + networks: + - laravel-development + depends_on: + php-fpm: + condition: service_started # Wait for php-fpm to start + + php-fpm: + # For the php-fpm service, we will use our common PHP-FPM Dockerfile with the development target + build: + context: . + dockerfile: ./docker/common/php-fpm/Dockerfile + target: development + args: + UID: ${UID:-1000} + GID: ${GID:-1000} + XDEBUG_ENABLED: ${XDEBUG_ENABLED:-true} + XDEBUG_MODE: develop,coverage,debug,profile + XDEBUG_HOST: ${XDEBUG_HOST:-host.docker.internal} + XDEBUG_IDE_KEY: ${XDEBUG_IDE_KEY:-DOCKER} + XDEBUG_LOG: /dev/stdout + XDEBUG_LOG_LEVEL: 0 + env_file: + # Load the environment variables from the Laravel application + - .env + user: "${UID:-1000}:${GID:-1000}" + volumes: + # Mount the application code for live updates + - ./:/var/www + networks: + - laravel-development + depends_on: + postgres: + condition: service_started # Wait for postgres to start + + workspace: + # For the workspace service, we will also create a custom image to install and setup all the necessary stuff. + build: + context: . + dockerfile: ./docker/development/workspace/Dockerfile + args: + UID: ${UID:-1000} + GID: ${GID:-1000} + XDEBUG_ENABLED: ${XDEBUG_ENABLED:-true} + XDEBUG_MODE: develop,coverage,debug,profile + XDEBUG_HOST: ${XDEBUG_HOST:-host.docker.internal} + XDEBUG_IDE_KEY: ${XDEBUG_IDE_KEY:-DOCKER} + XDEBUG_LOG: /dev/stdout + XDEBUG_LOG_LEVEL: 0 + tty: true # Enables an interactive terminal + stdin_open: true # Keeps standard input open for 'docker exec' + env_file: + - .env + volumes: + - ./:/var/www + networks: + - laravel-development + + postgres: + image: postgres:18 + ports: + - "${POSTGRES_PORT:-5432}:5432" + environment: + - POSTGRES_DB=app + - POSTGRES_USER=laravel + - POSTGRES_PASSWORD=secret + volumes: + - postgres-data-development:/var/lib/postgresql + networks: + - laravel-development + + redis: + image: redis:alpine + networks: + - laravel-development + +networks: + laravel-development: + +volumes: + postgres-data-development: +``` + +> [!NOTE] +> Ensure you have an `.env` file at the root of your Laravel project with the necessary configurations. You can use the `.env.example` file as a template. + +### Run your development environment + +To start the development environment, use: + +```console +$ docker compose -f compose.dev.yaml up --build -d +``` + +Run this command to build and start the development environment in detached mode. When the containers finish initializing, visit [http://localhost/](http://localhost/) to see your Laravel app in action. + +### Summary + +By building on top of the production image and adding debug tools like Xdebug, you create a Laravel development workflow that closely mirrors production. The optional workspace container simplifies tasks like asset building and running Artisan commands. If you prefer a separate container for every service (e.g., a dedicated `php-cli` and `node` container), you can skip the workspace approach. Either way, Docker Compose provides an efficient, consistent way to develop your Laravel project. + +## Common Questions on Using Laravel with Docker + + + +### 1. Why should I use Docker Compose for Laravel? + +Docker Compose is a powerful tool for managing multi-container environments, particularly in development due to its simplicity. With Docker Compose, you can define and connect all necessary services for Laravel, such as PHP, Nginx, and databases, in a single configuration (`compose.*.yaml`). This setup ensures consistency across development, testing, and production environments, streamlining onboarding and reducing discrepancies between local and server setups. + +While Docker Compose is a great choice for development, tools like **Docker Swarm** or **Kubernetes** offer advanced scaling and orchestration features, which may be beneficial for complex production deployments. + +### 2. How do I debug my Laravel application with Docker Compose? + +To debug your Laravel application in a Docker environment, use **Xdebug**. In the development setup, Xdebug is installed in the `php-fpm` container to enable debugging. Ensure Xdebug is enabled in your `compose.dev.yaml` file by setting the environment variable `XDEBUG_ENABLED=true` and configuring your IDE (e.g., Visual Studio Code or PHPStorm) to connect to the remote container for debugging. + +### 3. Can I use Docker Compose with databases other than PostgreSQL? + +Yes, Docker Compose supports various database services for Laravel. While PostgreSQL is used in the examples, you can easily substitute **MySQL**, **MariaDB**, or even **SQLite**. Update the `compose.*.yaml` file to specify the required Docker image and adjust your `.env` file to reflect the new database configuration. + +### 4. How can I persist data in development and production? + +In both development and production, Docker volumes are used to persist data. For instance, in the `compose.*.yaml` file, the `postgres-data-*` volume stores PostgreSQL data, ensuring that data is retained even if the container restarts. You can also define named volumes for other services where data persistence is essential. + +### 5. What is the difference between development and production Docker configurations? + +In a development environment, Docker configurations include tools that streamline coding and debugging, such as Xdebug for debugging, and volume mounts to enable real-time code updates without requiring image rebuilds. + +In production, the configuration is optimized for performance, security, and efficiency. This setup uses multi-stage builds to keep the image lightweight and includes only essential tools, packages, and libraries. + +It’s recommended to use `alpine`-based images in production for smaller image sizes, enhancing deployment speed and security. + +Additionally, consider using [Docker Scout](/manuals/scout/_index.md) to detect and analyze vulnerabilities, especially in production environments. + +For additional information about using Docker Compose in production, see [this guide](/compose/how-tos/production/). diff --git a/content/guides/frameworks/laravel/common-questions.md b/content/guides/frameworks/laravel/common-questions.md deleted file mode 100644 index 6fa01a3b414c..000000000000 --- a/content/guides/frameworks/laravel/common-questions.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -title: Common Questions on Using Laravel with Docker -description: Find answers to common questions about setting up and managing Laravel environments with Docker Compose, including troubleshooting and best practices. -keywords: laravel, php, docker compose, faq, troubleshooting, best practices -weight: 40 ---- - - - -## 1. Why should I use Docker Compose for Laravel? - -Docker Compose is a powerful tool for managing multi-container environments, particularly in development due to its simplicity. With Docker Compose, you can define and connect all necessary services for Laravel, such as PHP, Nginx, and databases, in a single configuration (`compose.*.yaml`). This setup ensures consistency across development, testing, and production environments, streamlining onboarding and reducing discrepancies between local and server setups. - -While Docker Compose is a great choice for development, tools like **Docker Swarm** or **Kubernetes** offer advanced scaling and orchestration features, which may be beneficial for complex production deployments. - -## 2. How do I debug my Laravel application with Docker Compose? - -To debug your Laravel application in a Docker environment, use **Xdebug**. In the development setup, Xdebug is installed in the `php-fpm` container to enable debugging. Ensure Xdebug is enabled in your `compose.dev.yaml` file by setting the environment variable `XDEBUG_ENABLED=true` and configuring your IDE (e.g., Visual Studio Code or PHPStorm) to connect to the remote container for debugging. - -## 3. Can I use Docker Compose with databases other than PostgreSQL? - -Yes, Docker Compose supports various database services for Laravel. While PostgreSQL is used in the examples, you can easily substitute **MySQL**, **MariaDB**, or even **SQLite**. Update the `compose.*.yaml` file to specify the required Docker image and adjust your `.env` file to reflect the new database configuration. - -## 4. How can I persist data in development and production? - -In both development and production, Docker volumes are used to persist data. For instance, in the `compose.*.yaml` file, the `postgres-data-*` volume stores PostgreSQL data, ensuring that data is retained even if the container restarts. You can also define named volumes for other services where data persistence is essential. - -## 5. What is the difference between development and production Docker configurations? - -In a development environment, Docker configurations include tools that streamline coding and debugging, such as Xdebug for debugging, and volume mounts to enable real-time code updates without requiring image rebuilds. - -In production, the configuration is optimized for performance, security, and efficiency. This setup uses multi-stage builds to keep the image lightweight and includes only essential tools, packages, and libraries. - -It’s recommended to use `alpine`-based images in production for smaller image sizes, enhancing deployment speed and security. - -Additionally, consider using [Docker Scout](/manuals/scout/_index.md) to detect and analyze vulnerabilities, especially in production environments. - -For additional information about using Docker Compose in production, see [this guide](/compose/how-tos/production/). diff --git a/content/guides/frameworks/laravel/development-setup.md b/content/guides/frameworks/laravel/development-setup.md deleted file mode 100644 index 1639612b1194..000000000000 --- a/content/guides/frameworks/laravel/development-setup.md +++ /dev/null @@ -1,328 +0,0 @@ ---- -title: Laravel Development Setup with Docker Compose -description: Set up a Laravel development environment using Docker Compose. -keywords: laravel, php, docker compose, development, xdebug, php-fpm, nginx -weight: 30 ---- - -This guide demonstrates how to configure a **development** environment for a Laravel application using Docker and Docker Compose. It builds **on top of** the production image for PHP-FPM and then adds developer-focused features—like Xdebug—to streamline debugging. By basing the development container on a known production image, you keep both environments closely aligned. - -This setup includes PHP-FPM, Nginx, and PostgreSQL services (although you can easily swap PostgreSQL for another database, like MySQL or MariaDB). Everything runs in containers, so you can develop in isolation without altering your host system. - -> [!NOTE] -> To experiment with a ready-to-run configuration, download the [Laravel Docker Examples](https://github.com/dockersamples/laravel-docker-examples) repository. It contains pre-configured setups for both development and production. - -## Project structure - -```plaintext -my-laravel-app/ -├── app/ -├── bootstrap/ -├── config/ -├── database/ -├── public/ -├── docker/ -│ ├── common/ -│ │ └── php-fpm/ -│ │ └── Dockerfile -│ ├── development/ -│ │ ├── php-fpm/ -│ │ │ └── entrypoint.sh -│ │ ├── workspace/ -│ │ │ └── Dockerfile -│ │ └── nginx -│ │ ├── Dockerfile -│ │ └── nginx.conf -│ └── production/ -├── compose.dev.yaml -├── compose.prod.yaml -├── .dockerignore -├── .env -├── vendor/ -├── ... -``` - -This layout represents a typical Laravel project, with Docker configurations stored in a unified `docker` directory. You’ll find **two** Compose files — `compose.dev.yaml` (for development) and `compose.prod.yaml` (for production) — to keep your environments separate and manageable. - -The environment includes a `workspace` service, a sidecar container for tasks like building front-end assets, running Artisan commands, and other CLI tools your project may require. While this extra container may seem unusual, it’s a familiar pattern in solutions like **Laravel Sail** and **Laradock**. It also includes **Xdebug** to aid in debugging. - -## Create a Dockerfile for PHP-FPM - -This Dockerfile **extends** the production image by installing Xdebug and adjusting user permissions to ease local development. That way, your development environment stays consistent with production while still offering extra debug features and improved file mounting. - -```dockerfile -# Builds a dev-only layer on top of the production image -FROM production AS development - -# Use ARGs to define environment variables passed from the Docker build command or Docker Compose. -ARG XDEBUG_ENABLED=true -ARG XDEBUG_MODE=develop,coverage,debug,profile -ARG XDEBUG_HOST=host.docker.internal -ARG XDEBUG_IDE_KEY=DOCKER -ARG XDEBUG_LOG=/dev/stdout -ARG XDEBUG_LOG_LEVEL=0 - -USER root - -# Configure Xdebug if enabled -RUN if [ "${XDEBUG_ENABLED}" = "true" ]; then \ - pecl install xdebug && \ - docker-php-ext-enable xdebug && \ - echo "xdebug.mode=${XDEBUG_MODE}" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini && \ - echo "xdebug.idekey=${XDEBUG_IDE_KEY}" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini && \ - echo "xdebug.log=${XDEBUG_LOG}" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini && \ - echo "xdebug.log_level=${XDEBUG_LOG_LEVEL}" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini && \ - echo "xdebug.client_host=${XDEBUG_HOST}" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini ; \ - echo "xdebug.start_with_request=yes" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini ; \ -fi - -# Add ARGs for syncing permissions -ARG UID=1000 -ARG GID=1000 - -# Create a new user with the specified UID and GID, reusing an existing group if GID exists -RUN if getent group ${GID}; then \ - group_name=$(getent group ${GID} | cut -d: -f1); \ - useradd -m -u ${UID} -g ${GID} -s /bin/bash www; \ - else \ - groupadd -g ${GID} www && \ - useradd -m -u ${UID} -g www -s /bin/bash www; \ - group_name=www; \ - fi - -# Dynamically update php-fpm to use the new user and group -RUN sed -i "s/user = www-data/user = www/g" /usr/local/etc/php-fpm.d/www.conf && \ - sed -i "s/group = www-data/group = $group_name/g" /usr/local/etc/php-fpm.d/www.conf - - -# Set the working directory -WORKDIR /var/www - -# Copy the entrypoint script -COPY ./docker/development/php-fpm/entrypoint.sh /usr/local/bin/entrypoint.sh -RUN chmod +x /usr/local/bin/entrypoint.sh - -# Switch back to the non-privileged user to run the application -USER www-data - -# Change the default command to run the entrypoint script -ENTRYPOINT ["/usr/local/bin/entrypoint.sh"] - -# Expose port 9000 and start php-fpm server -EXPOSE 9000 -CMD ["php-fpm"] -``` - -## Create a Dockerfile for Workspace - -A workspace container provides a dedicated shell for asset compilation, Artisan/Composer commands, and other CLI tasks. This approach follows patterns from Laravel Sail and Laradock, consolidating all development tools into one container for convenience. - -```dockerfile -# docker/development/workspace/Dockerfile -# Use the official PHP CLI image as the base -FROM php:8.5-cli - -# Set environment variables for user and group ID -ARG UID=1000 -ARG GID=1000 -ARG NODE_VERSION=22.0.0 - -# Install system dependencies and build libraries -RUN apt-get update && apt-get install -y --no-install-recommends \ - curl \ - unzip \ - libpq-dev \ - libonig-dev \ - libssl-dev \ - libxml2-dev \ - libcurl4-openssl-dev \ - libicu-dev \ - libzip-dev \ - && docker-php-ext-install -j$(nproc) \ - pdo_mysql \ - pdo_pgsql \ - pgsql \ - intl \ - zip \ - bcmath \ - soap \ - && pecl install redis \ - && docker-php-ext-enable redis \ - && curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer \ - && apt-get autoremove -y && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* - -# Use ARG to define environment variables passed from the Docker build command or Docker Compose. -ARG XDEBUG_ENABLED -ARG XDEBUG_MODE -ARG XDEBUG_HOST -ARG XDEBUG_IDE_KEY -ARG XDEBUG_LOG -ARG XDEBUG_LOG_LEVEL - -# Configure Xdebug if enabled -RUN if [ "${XDEBUG_ENABLED}" = "true" ]; then \ - pecl install xdebug && \ - docker-php-ext-enable xdebug && \ - echo "xdebug.mode=${XDEBUG_MODE}" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini && \ - echo "xdebug.idekey=${XDEBUG_IDE_KEY}" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini && \ - echo "xdebug.log=${XDEBUG_LOG}" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini && \ - echo "xdebug.log_level=${XDEBUG_LOG_LEVEL}" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini && \ - echo "xdebug.client_host=${XDEBUG_HOST}" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini ; \ - echo "xdebug.start_with_request=yes" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini ; \ -fi - -# If the group already exists, use it; otherwise, create the 'www' group -RUN if getent group ${GID}; then \ - useradd -m -u ${UID} -g ${GID} -s /bin/bash www; \ - else \ - groupadd -g ${GID} www && \ - useradd -m -u ${UID} -g www -s /bin/bash www; \ - fi && \ - usermod -aG sudo www && \ - echo 'www ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers - -# Switch to the non-root user to install NVM and Node.js -USER www - -# Install NVM (Node Version Manager) as the www user -RUN export NVM_DIR="$HOME/.nvm" && \ - curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.0/install.sh | bash && \ - [ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh" && \ - nvm install ${NODE_VERSION} && \ - nvm alias default ${NODE_VERSION} && \ - nvm use default - -# Ensure NVM is available for all future shells -RUN echo 'export NVM_DIR="$HOME/.nvm"' >> /home/www/.bashrc && \ - echo '[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"' >> /home/www/.bashrc && \ - echo '[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"' >> /home/www/.bashrc - -# Set the working directory -WORKDIR /var/www - -# Override the entrypoint to avoid the default php entrypoint -ENTRYPOINT [] - -# Default command to keep the container running -CMD ["bash"] -``` - -> [!NOTE] -> If you prefer a **one-service-per-container** approach, simply omit the workspace container and run separate containers for each task. For example, you could use a dedicated `php-cli` container for your PHP scripts, and a `node` container to handle the asset building. - -## Create a Docker Compose configuration for development - -Here's the `compose.yaml` file to set up the development environment: - -```yaml -services: - web: - image: nginx:latest # Using the default Nginx image with custom configuration. - volumes: - # Mount the application code for live updates - - ./:/var/www - # Mount the Nginx configuration file - - ./docker/development/nginx/nginx.conf:/etc/nginx/nginx.conf:ro - ports: - # Map port 80 inside the container to the port specified by 'NGINX_PORT' on the host machine - - "80:80" - environment: - - NGINX_HOST=localhost - networks: - - laravel-development - depends_on: - php-fpm: - condition: service_started # Wait for php-fpm to start - - php-fpm: - # For the php-fpm service, we will use our common PHP-FPM Dockerfile with the development target - build: - context: . - dockerfile: ./docker/common/php-fpm/Dockerfile - target: development - args: - UID: ${UID:-1000} - GID: ${GID:-1000} - XDEBUG_ENABLED: ${XDEBUG_ENABLED:-true} - XDEBUG_MODE: develop,coverage,debug,profile - XDEBUG_HOST: ${XDEBUG_HOST:-host.docker.internal} - XDEBUG_IDE_KEY: ${XDEBUG_IDE_KEY:-DOCKER} - XDEBUG_LOG: /dev/stdout - XDEBUG_LOG_LEVEL: 0 - env_file: - # Load the environment variables from the Laravel application - - .env - user: "${UID:-1000}:${GID:-1000}" - volumes: - # Mount the application code for live updates - - ./:/var/www - networks: - - laravel-development - depends_on: - postgres: - condition: service_started # Wait for postgres to start - - workspace: - # For the workspace service, we will also create a custom image to install and setup all the necessary stuff. - build: - context: . - dockerfile: ./docker/development/workspace/Dockerfile - args: - UID: ${UID:-1000} - GID: ${GID:-1000} - XDEBUG_ENABLED: ${XDEBUG_ENABLED:-true} - XDEBUG_MODE: develop,coverage,debug,profile - XDEBUG_HOST: ${XDEBUG_HOST:-host.docker.internal} - XDEBUG_IDE_KEY: ${XDEBUG_IDE_KEY:-DOCKER} - XDEBUG_LOG: /dev/stdout - XDEBUG_LOG_LEVEL: 0 - tty: true # Enables an interactive terminal - stdin_open: true # Keeps standard input open for 'docker exec' - env_file: - - .env - volumes: - - ./:/var/www - networks: - - laravel-development - - postgres: - image: postgres:18 - ports: - - "${POSTGRES_PORT:-5432}:5432" - environment: - - POSTGRES_DB=app - - POSTGRES_USER=laravel - - POSTGRES_PASSWORD=secret - volumes: - - postgres-data-development:/var/lib/postgresql - networks: - - laravel-development - - redis: - image: redis:alpine - networks: - - laravel-development - -networks: - laravel-development: - -volumes: - postgres-data-development: -``` - -> [!NOTE] -> Ensure you have an `.env` file at the root of your Laravel project with the necessary configurations. You can use the `.env.example` file as a template. - -## Run your development environment - -To start the development environment, use: - -```console -$ docker compose -f compose.dev.yaml up --build -d -``` - -Run this command to build and start the development environment in detached mode. When the containers finish initializing, visit [http://localhost/](http://localhost/) to see your Laravel app in action. - -## Summary - -By building on top of the production image and adding debug tools like Xdebug, you create a Laravel development workflow that closely mirrors production. The optional workspace container simplifies tasks like asset building and running Artisan commands. If you prefer a separate container for every service (e.g., a dedicated `php-cli` and `node` container), you can skip the workspace approach. Either way, Docker Compose provides an efficient, consistent way to develop your Laravel project. diff --git a/content/guides/frameworks/laravel/prerequisites.md b/content/guides/frameworks/laravel/prerequisites.md deleted file mode 100644 index cdf6e23e1edd..000000000000 --- a/content/guides/frameworks/laravel/prerequisites.md +++ /dev/null @@ -1,26 +0,0 @@ ---- -title: Prerequisites for Setting Up Laravel with Docker Compose -description: Ensure you have the required tools and knowledge before setting up Laravel with Docker Compose. -keywords: laravel, php, docker compose, prerequisites, requirements, setup -weight: 10 ---- - -Before you begin setting up Laravel with Docker Compose, make sure you meet the following prerequisites: - -## Docker and Docker Compose - -You need Docker and Docker Compose installed on your system. Docker allows you to containerize applications, and Docker Compose helps you manage multi-container applications. - -- Docker: Make sure Docker is installed and running on your machine. Refer to the [Docker installation guide](/get-docker/) to install Docker. -- Docker Compose: Docker Compose is included with Docker Desktop, but you can also follow the [Docker Compose installation guide](/compose/install/) if needed. - -## Basic understanding of Docker and containers - -A fundamental understanding of Docker and how containers work will be helpful. If you're new to Docker, consider reviewing the [Docker Overview](/get-started/overview/) to familiarize yourself with containerization concepts. - -## Basic knowledge of Laravel - -This guide assumes you have a basic understanding of Laravel and PHP. Familiarity with Laravel’s command-line tools, such as [Artisan](https://laravel.com/docs/12.x/artisan), and its project structure is important for following the instructions. - -- Laravel CLI: You should be comfortable using Laravel’s command-line tool (`artisan`). -- Laravel Project Structure: Familiarize yourself with Laravel’s folder structure (`app`, `config`, `routes`, `tests`, etc.). diff --git a/content/guides/frameworks/laravel/production-setup.md b/content/guides/frameworks/laravel/production-setup.md deleted file mode 100644 index 9e04125093d7..000000000000 --- a/content/guides/frameworks/laravel/production-setup.md +++ /dev/null @@ -1,444 +0,0 @@ ---- -title: Laravel Production Setup with Docker Compose -description: Set up a production-ready environment for Laravel using Docker Compose. -keywords: laravel, php, docker compose, production, deployment, php-fpm -weight: 20 ---- - -This guide demonstrates how to set up a production-ready Laravel environment using Docker and Docker Compose. This configuration is designed for streamlined, scalable, and secure Laravel application deployments. - -> [!NOTE] -> To experiment with a ready-to-run configuration, download the [Laravel Docker Examples](https://github.com/dockersamples/laravel-docker-examples) repository. It contains pre-configured setups for both development and production. - -## Project structure - -```plaintext -my-laravel-app/ -├── app/ -├── bootstrap/ -├── config/ -├── database/ -├── public/ -├── docker/ -│ ├── common/ -│ │ └── php-fpm/ -│ │ ├── Dockerfile -│ │ └── conf.d/ -│ │ └── 20-status-path.conf -│ ├── development/ -│ ├── production/ -│ │ ├── php-fpm/ -│ │ │ └── entrypoint.sh -│ │ └── nginx -│ │ ├── Dockerfile -│ │ └── nginx.conf -├── compose.dev.yaml -├── compose.prod.yaml -├── .dockerignore -├── .env -├── vendor/ -├── ... -``` - -This layout represents a typical Laravel project, with Docker configurations stored in a unified `docker` directory. You’ll find **two** Compose files — `compose.dev.yaml` (for development) and `compose.prod.yaml` (for production) — to keep your environments separate and manageable. - -## Create a Dockerfile for PHP-FPM (production) - -For production, the `php-fpm` Dockerfile creates an optimized image with only the PHP extensions and libraries your application needs. As demonstrated in the [GitHub example](https://github.com/dockersamples/laravel-docker-examples), a single Dockerfile with multi-stage builds maintains consistency and reduces duplication between development and production. The following snippet shows only the production-related stages: - -```dockerfile -# Stage 1: Build environment and Composer dependencies -FROM php:8.5-fpm AS builder - -# Install system dependencies and PHP extensions for Laravel with MySQL/PostgreSQL support. -# Dependencies in this stage are only required for building the final image. -# Node.js and asset building are handled in the Nginx stage, not here. -RUN apt-get update && apt-get install -y --no-install-recommends \ - curl \ - unzip \ - libpq-dev \ - libonig-dev \ - libssl-dev \ - libxml2-dev \ - libcurl4-openssl-dev \ - libicu-dev \ - libzip-dev \ - && docker-php-ext-install -j$(nproc) \ - pdo_mysql \ - pdo_pgsql \ - pgsql \ - intl \ - zip \ - bcmath \ - soap \ - && pecl install redis \ - && docker-php-ext-enable redis \ - && apt-get autoremove -y && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* - -# Set the working directory inside the container -WORKDIR /var/www - -# Copy the entire Laravel application code into the container -# ----------------------------------------------------------- -# In Laravel, `composer install` may trigger scripts -# needing access to application code. -# For example, the `post-autoload-dump` event might execute -# Artisan commands like `php artisan package:discover`. If the -# application code (including the `artisan` file) is not -# present, these commands will fail, leading to build errors. -# -# By copying the entire application code before running -# `composer install`, we ensure that all necessary files are -# available, allowing these scripts to run successfully. -# In other cases, it would be possible to copy composer files -# first, to leverage Docker's layer caching mechanism. -# ----------------------------------------------------------- -COPY . /var/www - -# Install Composer and dependencies -RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer \ - && composer install --no-dev --optimize-autoloader --no-interaction --no-progress --prefer-dist - -# Stage 2: Production environment -FROM php:8.5-fpm AS production - -# Install only runtime libraries needed in production -# libfcgi-bin and procps are required for the php-fpm-healthcheck script -RUN apt-get update && apt-get install -y --no-install-recommends \ - libpq-dev \ - libicu-dev \ - libzip-dev \ - libfcgi-bin \ - procps \ - && apt-get autoremove -y && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* - -# Download and install php-fpm health check script -RUN curl -o /usr/local/bin/php-fpm-healthcheck \ - https://raw.githubusercontent.com/renatomefi/php-fpm-healthcheck/master/php-fpm-healthcheck \ - && chmod +x /usr/local/bin/php-fpm-healthcheck - -# Copy the initialization script -COPY ./docker/php-fpm/entrypoint.sh /usr/local/bin/entrypoint.sh -RUN chmod +x /usr/local/bin/entrypoint.sh - -# Copy the initial storage structure -COPY ./storage /var/www/storage-init - -# Copy PHP extensions and libraries from the builder stage -COPY --from=builder /usr/local/lib/php/extensions/ /usr/local/lib/php/extensions/ -COPY --from=builder /usr/local/etc/php/conf.d/ /usr/local/etc/php/conf.d/ -COPY --from=builder /usr/local/bin/docker-php-ext-* /usr/local/bin/ - -# Use the recommended production PHP configuration -# ----------------------------------------------------------- -# PHP provides development and production configurations. -# Here, we replace the default php.ini with the production -# version to apply settings optimized for performance and -# security in a live environment. -# ----------------------------------------------------------- -RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini" - -# Keep the image-provided FPM global config intact and add pool overrides separately -COPY ./docker/common/php-fpm/conf.d/*.conf /usr/local/etc/php-fpm.d/ -# Update the variables_order to include E (for ENV) -#RUN sed -i 's/variables_order = "GPCS"/variables_order = "EGPCS"/' "$PHP_INI_DIR/php.ini" - -# Copy the application code and dependencies from the build stage -COPY --from=builder /var/www /var/www - -# Set working directory -WORKDIR /var/www - -# Ensure correct permissions -RUN chown -R www-data:www-data /var/www - -# Switch to the non-privileged user to run the application -USER www-data - -# Change the default command to run the entrypoint script -ENTRYPOINT ["/usr/local/bin/entrypoint.sh"] - -# Expose port 9000 and start php-fpm server -EXPOSE 9000 -CMD ["php-fpm"] -``` - -## Create a Dockerfile for PHP-CLI (production) - -For production, you often need a separate container to run Artisan commands, migrations, and other CLI tasks. In most cases you can run these commands by reusing existing PHP-FPM container: - -```console -$ docker compose -f compose.prod.yaml exec php-fpm php artisan route:list -``` - -If you need a separate CLI container with different extensions or strict separation of concerns, consider a php-cli Dockerfile: - -```dockerfile -# Stage 1: Build environment and Composer dependencies -FROM php:8.5-cli AS builder - -# Install system dependencies and PHP extensions required for Laravel + MySQL/PostgreSQL support -# Some dependencies are required for PHP extensions only in the build stage -RUN apt-get update && apt-get install -y --no-install-recommends \ - curl \ - unzip \ - libpq-dev \ - libonig-dev \ - libssl-dev \ - libxml2-dev \ - libcurl4-openssl-dev \ - libicu-dev \ - libzip-dev \ - && docker-php-ext-install -j$(nproc) \ - pdo_mysql \ - pdo_pgsql \ - pgsql \ - intl \ - zip \ - bcmath \ - soap \ - && pecl install redis \ - && docker-php-ext-enable redis \ - && apt-get autoremove -y && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* - -# Set the working directory inside the container -WORKDIR /var/www - -# Copy the entire Laravel application code into the container -COPY . /var/www - -# Install Composer and dependencies -RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer \ - && composer install --no-dev --optimize-autoloader --no-interaction --no-progress --prefer-dist - -# Stage 2: Production environment -FROM php:8.5-cli - -# Install client libraries required for php extensions in runtime -RUN apt-get update && apt-get install -y --no-install-recommends \ - libpq-dev \ - libicu-dev \ - libzip-dev \ - && apt-get autoremove -y && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* - -# Copy PHP extensions and libraries from the builder stage -COPY --from=builder /usr/local/lib/php/extensions/ /usr/local/lib/php/extensions/ -COPY --from=builder /usr/local/etc/php/conf.d/ /usr/local/etc/php/conf.d/ -COPY --from=builder /usr/local/bin/docker-php-ext-* /usr/local/bin/ - -# Use the default production configuration for PHP runtime arguments -RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini" - -# Copy the application code and dependencies from the build stage -COPY --from=builder /var/www /var/www - -# Set working directory -WORKDIR /var/www - -# Ensure correct permissions -RUN chown -R www-data:www-data /var/www - -# Switch to the non-privileged user to run the application -USER www-data - -# Default command: Provide a bash shell to allow running any command -CMD ["bash"] -``` - -This Dockerfile is similar to the PHP-FPM Dockerfile, but it uses the `php:8.5-cli` image as the base image and sets up the container for running CLI commands. - -## Create a Dockerfile for Nginx (production) - -Nginx serves as the web server for the Laravel application. You can include static assets directly to the container. Here's an example of possible Dockerfile for Nginx: - -```dockerfile -# docker/nginx/Dockerfile -# Stage 1: Build assets -FROM debian AS builder - -# Install Node.js and build tools -RUN apt-get update && apt-get install -y --no-install-recommends \ - curl \ - nodejs \ - npm \ - && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* - -# Set working directory -WORKDIR /var/www - -# Copy Laravel application code -COPY . /var/www - -# Install Node.js dependencies and build assets -RUN npm install && npm run build - -# Stage 2: Nginx production image -FROM nginx:alpine - -# Copy custom Nginx configuration -# ----------------------------------------------------------- -# Replace the default Nginx configuration with our custom one -# that is optimized for serving a Laravel application. -# ----------------------------------------------------------- -COPY ./docker/nginx/nginx.conf /etc/nginx/nginx.conf - -# Copy Laravel's public assets from the builder stage -# ----------------------------------------------------------- -# We only need the 'public' directory from our Laravel app. -# ----------------------------------------------------------- -COPY --from=builder /var/www/public /var/www/public - -# Set the working directory to the public folder -WORKDIR /var/www/public - -# Expose port 80 and start Nginx -EXPOSE 80 -CMD ["nginx", "-g", "daemon off;"] -``` - -This Dockerfile uses a multi-stage build to separate the asset building process from the final production image. The first stage installs Node.js and builds the assets, while the second stage sets up the Nginx production image with the optimized configuration and the built assets. - -## Create a Docker Compose configuration for production - -To bring all the services together, create a `compose.prod.yaml` file that defines the services, volumes, and networks for the production environment. Here's an example configuration: - -```yaml -services: - web: - build: - context: . - dockerfile: ./docker/production/nginx/Dockerfile - restart: unless-stopped # Automatically restart unless the service is explicitly stopped - volumes: - # Mount the 'laravel-storage' volume to '/var/www/storage' inside the container. - # ----------------------------------------------------------- - # This volume stores persistent data like uploaded files and cache. - # The ':ro' option mounts it as read-only in the 'web' service because Nginx only needs to read these files. - # The 'php-fpm' service mounts the same volume without ':ro' to allow write operations. - # ----------------------------------------------------------- - - laravel-storage-production:/var/www/storage:ro - networks: - - laravel-production - ports: - # Map port 80 inside the container to the port specified by 'NGINX_PORT' on the host machine. - # ----------------------------------------------------------- - # This allows external access to the Nginx web server running inside the container. - # For example, if 'NGINX_PORT' is set to '8080', accessing 'http://localhost:8080' will reach the application. - # ----------------------------------------------------------- - - "${NGINX_PORT:-80}:80" - depends_on: - php-fpm: - condition: service_healthy # Wait for php-fpm health check - - php-fpm: - # For the php-fpm service, we will create a custom image to install the necessary PHP extensions and setup proper permissions. - build: - context: . - dockerfile: ./docker/common/php-fpm/Dockerfile - target: production # Use the 'production' stage in the Dockerfile - restart: unless-stopped - volumes: - - laravel-storage-production:/var/www/storage # Mount the storage volume - env_file: - - .env - networks: - - laravel-production - healthcheck: - test: ["CMD-SHELL", "php-fpm-healthcheck || exit 1"] - interval: 10s - timeout: 5s - retries: 3 - # The 'depends_on' attribute with 'condition: service_healthy' ensures that - # this service will not start until the 'postgres' service passes its health check. - # This prevents the application from trying to connect to the database before it's ready. - depends_on: - postgres: - condition: service_healthy - - # The 'php-cli' service provides a command-line interface for running Artisan commands and other CLI tasks. - # ----------------------------------------------------------- - # This is useful for running migrations, seeders, or any custom scripts. - # It shares the same codebase and environment as the 'php-fpm' service. - # ----------------------------------------------------------- - php-cli: - build: - context: . - dockerfile: ./docker/php-cli/Dockerfile - tty: true # Enables an interactive terminal - stdin_open: true # Keeps standard input open for 'docker exec' - env_file: - - .env - networks: - - laravel-production - - postgres: - image: postgres:18 - restart: unless-stopped - user: postgres - ports: - - "${POSTGRES_PORT}:5432" - environment: - - POSTGRES_DB=${POSTGRES_DATABASE} - - POSTGRES_USER=${POSTGRES_USERNAME} - - POSTGRES_PASSWORD=${POSTGRES_PASSWORD} - volumes: - - postgres-data-production:/var/lib/postgresql - networks: - - laravel-production - # Health check for PostgreSQL - # ----------------------------------------------------------- - # Health checks allow Docker to determine if a service is operational. - # The 'pg_isready' command checks if PostgreSQL is ready to accept connections. - # This prevents dependent services from starting before the database is ready. - # ----------------------------------------------------------- - healthcheck: - test: ["CMD", "pg_isready"] - interval: 10s - timeout: 5s - retries: 5 - - redis: - image: redis:alpine - restart: unless-stopped # Automatically restart unless the service is explicitly stopped - networks: - - laravel-production - # Health check for Redis - # ----------------------------------------------------------- - # Checks if Redis is responding to the 'PING' command. - # This ensures that the service is not only running but also operational. - # ----------------------------------------------------------- - healthcheck: - test: ["CMD", "redis-cli", "ping"] - interval: 10s - timeout: 5s - retries: 3 - -networks: - # Attach the service to the 'laravel-production' network. - # ----------------------------------------------------------- - # This custom network allows all services within it to communicate using their service names as hostnames. - # For example, 'php-fpm' can connect to 'postgres' by using 'postgres' as the hostname. - # ----------------------------------------------------------- - laravel-production: - -volumes: - postgres-data-production: - laravel-storage-production: -``` - -> [!NOTE] -> Ensure you have an `.env` file at the root of your Laravel project with the necessary configurations to match the Docker Compose setup. - -## Running your production environment - -To start the production environment, run: - -```console -$ docker compose -f compose.prod.yaml up --build -d -``` - -This command will build and start all the services in detached mode, providing a scalable and production-ready setup for your Laravel application. - -## Summary - -By setting up a Docker Compose environment for Laravel in production, you ensure that your application is optimized for performance, scalable, and secure. This setup makes deployments consistent and easier to manage, reducing the likelihood of errors due to differences between environments. diff --git a/content/guides/genai-claude-code-mcp/claude-code-mcp-guide.md b/content/guides/genai-claude-code-mcp/claude-code-mcp-guide.md index 22624c6bd3c8..471b42f7231f 100644 --- a/content/guides/genai-claude-code-mcp/claude-code-mcp-guide.md +++ b/content/guides/genai-claude-code-mcp/claude-code-mcp-guide.md @@ -5,10 +5,10 @@ title: Generate Docker Compose Files with Claude Code and Docker MCP Toolkit summary: | This guide shows how to wire Claude Code to the Docker MCP Toolkit so it can search Docker Hub images and generate complete Docker Compose stacks from natural language. You’ll enable the Docker Hub MCP server, connect Claude Code, verify MCP access, and create a Node.js + PostgreSQL stack with a conversational prompt. -tags: [ai] aliases: - /guides/use-case/genai-claude-code-mcp/ params: + tags: [ai] time: 15 minutes --- diff --git a/content/guides/genai-leveraging-rag/index.md b/content/guides/genai-leveraging-rag/index.md index b3194e7c2727..a687258b4485 100644 --- a/content/guides/genai-leveraging-rag/index.md +++ b/content/guides/genai-leveraging-rag/index.md @@ -5,8 +5,8 @@ description: This guide walks through the process of setting up and utilizing a keywords: Docker, GenAI, Retrieval-Augmented Generation, RAG, Graph Databases, Neo4j, AI, LLM summary: | This guide explains setting up a GenAI stack with Retrieval-Augmented Generation (RAG) and Neo4j, covering key concepts, deployment steps, and a case study. It also includes troubleshooting tips for optimizing AI performance with real-time data. -tags: [ai] params: + tags: [ai] time: 35 minutes --- diff --git a/content/guides/genai-pdf-bot/_index.md b/content/guides/genai-pdf-bot/_index.md index 41c5e16626e7..469055455934 100644 --- a/content/guides/genai-pdf-bot/_index.md +++ b/content/guides/genai-pdf-bot/_index.md @@ -5,16 +5,530 @@ keywords: python, generative ai, genai, llm, neo4j, ollama, langchain summary: | Learn how to build a PDF bot for parsing PDF documents and generating responses using Docker and generative AI. -tags: [ai] aliases: - /guides/use-case/genai-pdf-bot/ + - /guides/genai-pdf-bot/containerize/ + - /guides/genai-pdf-bot/develop/ params: + tags: [ai] time: 20 minutes --- + The generative AI (GenAI) guide teaches you how to containerize an existing GenAI application using Docker. In this guide, you’ll learn how to: - Containerize and run a Python-based GenAI application - Set up a local environment to run the complete GenAI stack locally for development Start by containerizing an existing GenAI application. + +## Containerize a generative AI application + +### Prerequisites + +> [!NOTE] +> +> GenAI applications can often benefit from GPU acceleration. Currently Docker Desktop supports GPU acceleration only on [Windows with the WSL2 backend](/manuals/desktop/features/gpu.md#using-nvidia-gpus-with-wsl2). Linux users can also access GPU acceleration using a native installation of the [Docker Engine](/manuals/engine/install/_index.md). + +- You have installed the latest version of [Docker Desktop](/get-started/get-docker.md) or, if you are a Linux user and are planning to use GPU acceleration, [Docker Engine](/manuals/engine/install/_index.md). Docker adds new features regularly and some parts of this guide may work only with the latest version of Docker Desktop. +- You have a [git client](https://git-scm.com/downloads). The examples in this section use a command-line based git client, but you can use any client. + +### Overview + +This section walks you through containerizing a generative AI (GenAI) application using Docker Desktop. + +> [!NOTE] +> +> You can see more samples of containerized GenAI applications in the [GenAI Stack](https://github.com/docker/genai-stack) demo applications. + +### Get the sample application + +The sample application used in this guide is a modified version of the PDF Reader application from the [GenAI Stack](https://github.com/docker/genai-stack) demo applications. The application is a full stack Python application that lets you ask questions about a PDF file. + +The application uses [LangChain](https://www.langchain.com/) for orchestration, [Streamlit](https://streamlit.io/) for the UI, [Ollama](https://ollama.ai/) to run the LLM, and [Neo4j](https://neo4j.com/) to store vectors. + +Clone the sample application. Open a terminal, change directory to a directory that you want to work in, and run the following command to clone the repository: + +```console +$ git clone https://github.com/craig-osterhout/docker-genai-sample +``` + +You should now have the following files in your `docker-genai-sample` directory. + +```text +├── docker-genai-sample/ +│ ├── .gitignore +│ ├── app.py +│ ├── chains.py +│ ├── env.example +│ ├── requirements.txt +│ ├── util.py +│ ├── LICENSE +│ └── README.md +``` + +### Create Docker assets + +Now that you have an application, you can create the necessary Docker assets to +containerize it. + +> [!TIP] +> +> [Gordon](/ai/gordon/), Docker's AI assistant, can generate Docker assets for your project. Ask Gordon to create a Dockerfile, Compose file, and `.dockerignore` tailored to your application. + +Create the following files in your `docker-genai-sample` directory. + +```dockerfile {collapse=true,title=Dockerfile} +# syntax=docker/dockerfile:1 + +# Comments are provided throughout this file to help you get started. +# If you need more help, visit the Dockerfile reference guide at +# https://docs.docker.com/go/dockerfile-reference/ + +ARG PYTHON_VERSION=3.11.4 +FROM python:${PYTHON_VERSION}-slim as base + +# Prevents Python from writing pyc files. +ENV PYTHONDONTWRITEBYTECODE=1 + +# Keeps Python from buffering stdout and stderr to avoid situations where +# the application crashes without emitting any logs due to buffering. +ENV PYTHONUNBUFFERED=1 + +WORKDIR /app + +# Create a non-privileged user that the app will run under. +# See https://docs.docker.com/go/dockerfile-user-best-practices/ +ARG UID=10001 +RUN adduser \ + --disabled-password \ + --gecos "" \ + --home "/nonexistent" \ + --shell "/sbin/nologin" \ + --no-create-home \ + --uid "${UID}" \ + appuser + +# Download dependencies as a separate step to take advantage of Docker's caching. +# Leverage a cache mount to /root/.cache/pip to speed up subsequent builds. +# Leverage a bind mount to requirements.txt to avoid having to copy them into +# into this layer. +RUN --mount=type=cache,target=/root/.cache/pip \ + --mount=type=bind,source=requirements.txt,target=requirements.txt \ + python -m pip install -r requirements.txt + +# Switch to the non-privileged user to run the application. +USER appuser + +# Copy the source code into the container. +COPY . . + +# Expose the port that the application listens on. +EXPOSE 8000 + +# Run the application. +CMD ["streamlit", "run", "app.py", "--server.address=0.0.0.0", "--server.port=8000"] +``` + +```yaml {collapse=true,title=compose.yaml} +# Comments are provided throughout this file to help you get started. +# If you need more help, visit the Docker Compose reference guide at +# https://docs.docker.com/go/compose-spec-reference/ + +# Here the instructions define your application as a service called "server". +# This service is built from the Dockerfile in the current directory. +# You can add other services your application may depend on here, such as a +# database or a cache. For examples, see the Awesome Compose repository: +# https://github.com/docker/awesome-compose +services: + server: + build: + context: . + ports: + - 8000:8000 + +# The commented out section below is an example of how to define a PostgreSQL +# database that your application can use. `depends_on` tells Docker Compose to +# start the database before your application. The `db-data` volume persists the +# database data between container restarts. The `db-password` secret is used +# to set the database password. You must create `db/password.txt` and add +# a password of your choosing to it before running `docker compose up`. +# depends_on: +# db: +# condition: service_healthy +# db: +# image: postgres +# restart: always +# user: postgres +# secrets: +# - db-password +# volumes: +# - db-data:/var/lib/postgresql/data +# environment: +# - POSTGRES_DB=example +# - POSTGRES_PASSWORD_FILE=/run/secrets/db-password +# expose: +# - 5432 +# healthcheck: +# test: [ "CMD", "pg_isready" ] +# interval: 10s +# timeout: 5s +# retries: 5 +# volumes: +# db-data: +# secrets: +# db-password: +# file: db/password.txt +``` + +```text {collapse=true,title=".dockerignore"} +# Include any files or directories that you don't want to be copied to your +# container here (e.g., local build artifacts, temporary files, etc.). +# +# For more help, visit the .dockerignore file reference guide at +# https://docs.docker.com/go/build-context-dockerignore/ + +**/.DS_Store +**/__pycache__ +**/.venv +**/.classpath +**/.dockerignore +**/.env +**/.git +**/.gitignore +**/.project +**/.settings +**/.toolstarget +**/.vs +**/.vscode +**/*.*proj.user +**/*.dbmdl +**/*.jfm +**/bin +**/charts +**/docker-compose* +**/compose.y*ml +**/Dockerfile* +**/node_modules +**/npm-debug.log +**/obj +**/secrets.dev.yaml +**/values.dev.yaml +LICENSE +README.md +``` + +You should now have the following contents in your `docker-genai-sample` +directory. + +```text +├── docker-genai-sample/ +│ ├── .dockerignore +│ ├── .gitignore +│ ├── app.py +│ ├── chains.py +│ ├── compose.yaml +│ ├── env.example +│ ├── requirements.txt +│ ├── util.py +│ ├── Dockerfile +│ ├── LICENSE +│ └── README.md +``` + +To learn more about these files, see the following: + +- [Dockerfile](../../../reference/dockerfile.md) +- [.dockerignore](../../../reference/dockerfile.md#dockerignore-file) +- [compose.yaml](/reference/compose-file/_index.md) + +### Run the application + +Inside the `docker-genai-sample` directory, run the following command in a +terminal. + +```console +$ docker compose up --build +``` + +Docker builds and runs your application. Depending on your network connection, it may take several minutes to download all the dependencies. You'll see a message like the following in the terminal when the application is running. + +```console +server-1 | You can now view your Streamlit app in your browser. +server-1 | +server-1 | URL: http://0.0.0.0:8000 +server-1 | +``` + +Open a browser and view the application at [http://localhost:8000](http://localhost:8000). You should see a simple Streamlit application. The application may take a few minutes to download the embedding model. While the download is in progress, **Running** appears in the top-right corner. + +The application requires a Neo4j database service and an LLM service to +function. If you have access to services that you ran outside of Docker, specify +the connection information and try it out. If you don't have the services +running, continue with this guide to learn how you can run some or all of these +services with Docker. + +In the terminal, press `ctrl`+`c` to stop the application. + +### Summary + +In this section, you learned how you can containerize and run your GenAI +application using Docker. + +### Next steps + +In the next section, you'll learn how you can run your application, database, and LLM service all locally using Docker. + +## Use containers for generative AI development + +### Prerequisites + +Complete [Containerize a generative AI application](containerize.md). + +### Overview + +In this section, you'll learn how to set up a development environment to access all the services that your generative AI (GenAI) application needs. This includes: + +- Adding a local database +- Adding a local or remote LLM service + +> [!NOTE] +> +> You can see more samples of containerized GenAI applications in the [GenAI Stack](https://github.com/docker/genai-stack) demo applications. + +### Add a local database + +You can use containers to set up local services, like a database. In this section, you'll update the `compose.yaml` file to define a database service. In addition, you'll specify an environment variables file to load the database connection information rather than manually entering the information every time. + +To run the database service: + +1. In the cloned repository's directory, rename `env.example` file to `.env`. + This file contains the environment variables that the containers will use. +2. In the cloned repository's directory, open the `compose.yaml` file in an IDE or text editor. +3. In the `compose.yaml` file, add the following: + + - Add instructions to run a Neo4j database + - Specify the environment file under the server service in order to pass in the environment variables for the connection + + The following is the updated `compose.yaml` file. All comments have been removed. + + ```yaml{hl_lines=["7-23"]} + services: + server: + build: + context: . + ports: + - 8000:8000 + env_file: + - .env + depends_on: + database: + condition: service_healthy + database: + image: neo4j:5.11 + ports: + - "7474:7474" + - "7687:7687" + environment: + - NEO4J_AUTH=${NEO4J_USERNAME}/${NEO4J_PASSWORD} + healthcheck: + test: ["CMD-SHELL", "wget --no-verbose --tries=1 --spider localhost:7474 || exit 1"] + interval: 5s + timeout: 3s + retries: 5 + ``` + + > [!NOTE] + > + > To learn more about Neo4j, see the [Neo4j Official Docker Image](https://hub.docker.com/_/neo4j). + +4. Run the application. Inside the `docker-genai-sample` directory, + run the following command in a terminal. + + ```console + $ docker compose up --build + ``` + +5. Access the application. Open a browser and view the application at [http://localhost:8000](http://localhost:8000). You should see a simple Streamlit application. Note that asking questions to a PDF will cause the application to fail because the LLM service specified in the `.env` file isn't running yet. + +6. Stop the application. In the terminal, press `ctrl`+`c` to stop the application. + +### Add a local or remote LLM service + +The sample application supports both [Ollama](https://ollama.ai/) and [OpenAI](https://openai.com/). This guide provides instructions for the following scenarios: + +- Run Ollama in a container +- Run Ollama outside of a container +- Use OpenAI + +While all platforms can use any of the previous scenarios, the performance and +GPU support may vary. You can use the following guidelines to help you choose the appropriate option: + +- Run Ollama in a container if you're on Linux, and using a native installation of the Docker Engine, or Windows 10/11, and using Docker Desktop, you + have a CUDA-supported GPU, and your system has at least 8 GB of RAM. +- Run Ollama outside of a container if you're on an Apple silicon Mac. +- Use OpenAI if the previous two scenarios don't apply to you. + +Choose one of the following options for your LLM service. + +{{< tabs >}} +{{< tab name="Run Ollama in a container" >}} + +When running Ollama in a container, you should have a CUDA-supported GPU. While you can run Ollama in a container without a supported GPU, the performance may not be acceptable. Only Linux and Windows 11 support GPU access to containers. + +To run Ollama in a container and provide GPU access: + +1. Install the prerequisites. + - For Docker Engine on Linux, install the [NVIDIA Container Toolkit](https://github.com/NVIDIA/nvidia-container-toolkit). + - For Docker Desktop on Windows 10/11, install the latest [NVIDIA driver](https://www.nvidia.com/Download/index.aspx) and make sure you are using the [WSL2 backend](/manuals/desktop/features/wsl/_index.md#turn-on-docker-desktop-wsl-2) +2. Add the Ollama service and a volume in your `compose.yaml`. The following is + the updated `compose.yaml`: + + ```yaml {hl_lines=["24-38"]} + services: + server: + build: + context: . + ports: + - 8000:8000 + env_file: + - .env + depends_on: + database: + condition: service_healthy + database: + image: neo4j:5.11 + ports: + - "7474:7474" + - "7687:7687" + environment: + - NEO4J_AUTH=${NEO4J_USERNAME}/${NEO4J_PASSWORD} + healthcheck: + test: + [ + "CMD-SHELL", + "wget --no-verbose --tries=1 --spider localhost:7474 || exit 1", + ] + interval: 5s + timeout: 3s + retries: 5 + ollama: + image: ollama/ollama:latest + ports: + - "11434:11434" + volumes: + - ollama_volume:/root/.ollama + deploy: + resources: + reservations: + devices: + - driver: nvidia + count: all + capabilities: [gpu] + volumes: + ollama_volume: + ``` + + > [!NOTE] + > + > For more details about the Compose instructions, see [Turn on GPU access with Docker Compose](/manuals/compose/how-tos/gpu-support.md). + +3. Add the ollama-pull service to your `compose.yaml` file. This service uses + the `docker/genai:ollama-pull` image, based on the GenAI Stack's + [pull_model.Dockerfile](https://github.com/docker/genai-stack/blob/main/pull_model.Dockerfile). + The service will automatically pull the model for your Ollama + container. The following is the updated section of the `compose.yaml` file: + + ```yaml {hl_lines=["12-17"]} + services: + server: + build: + context: . + ports: + - 8000:8000 + env_file: + - .env + depends_on: + database: + condition: service_healthy + ollama-pull: + condition: service_completed_successfully + ollama-pull: + image: docker/genai:ollama-pull + env_file: + - .env + # ... + ``` + +{{< /tab >}} +{{< tab name="Run Ollama outside of a container" >}} + +To run Ollama outside of a container: + +1. [Install](https://github.com/jmorganca/ollama) and run Ollama on your host + machine. +2. Update the `OLLAMA_BASE_URL` value in your `.env` file to + `http://host.docker.internal:11434`. +3. Pull the model to Ollama using the following command. + ```console + $ ollama pull llama2 + ``` + +{{< /tab >}} +{{< tab name="Use OpenAI" >}} + +> [!IMPORTANT] +> +> Using OpenAI requires an [OpenAI account](https://platform.openai.com/login). OpenAI is a third-party hosted service and charges may apply. + +1. Update the `LLM` value in your `.env` file to + `gpt-3.5`. +2. Uncomment and update the `OPENAI_API_KEY` value in your `.env` file to + your [OpenAI API key](https://help.openai.com/en/articles/4936850-where-do-i-find-my-api-key). + +{{< /tab >}} +{{< /tabs >}} + +### Run your GenAI application + +At this point, you have the following services in your Compose file: + +- Server service for your main GenAI application +- Database service to store vectors in a Neo4j database +- (optional) Ollama service to run the LLM +- (optional) Ollama-pull service to automatically pull the model for the Ollama + service + +To run all the services, run the following command in your `docker-genai-sample` +directory: + +```console +$ docker compose up --build +``` + +If your Compose file has the ollama-pull service, it may take several minutes for the ollama-pull service to pull the model. The ollama-pull service will continuously update the console with its status. After pulling the model, the ollama-pull service container will stop and you can access the application. + +Once the application is running, open a browser and access the application at [http://localhost:8000](http://localhost:8000). + +Upload a PDF file, for example the [Docker CLI Cheat Sheet](https://docs.docker.com/get-started/docker_cheatsheet.pdf), and ask a question about the PDF. + +Depending on your system and the LLM service that you chose, it may take several +minutes to answer. If you are using Ollama and the performance isn't +acceptable, try using OpenAI. + +### Summary + +In this section, you learned how to set up a development environment to provide +access all the services that your GenAI application needs. + +Related information: + +- [Dockerfile reference](../../../reference/dockerfile.md) +- [Compose file reference](/reference/compose-file/_index.md) +- [Ollama Docker image](https://hub.docker.com/r/ollama/ollama) +- [Neo4j Official Docker Image](https://hub.docker.com/_/neo4j) +- [GenAI Stack demo applications](https://github.com/docker/genai-stack) + +### Next steps + +See samples of more GenAI applications in the [GenAI Stack demo applications](https://github.com/docker/genai-stack). diff --git a/content/guides/genai-pdf-bot/containerize.md b/content/guides/genai-pdf-bot/containerize.md deleted file mode 100644 index 93aed280e965..000000000000 --- a/content/guides/genai-pdf-bot/containerize.md +++ /dev/null @@ -1,264 +0,0 @@ ---- -title: Containerize a generative AI application -linkTitle: Containerize your app -weight: 10 -keywords: python, generative ai, genai, llm, neo4j, ollama, containerize, initialize, langchain, openai -description: Learn how to containerize a generative AI (GenAI) application. -aliases: - - /guides/use-case/genai-pdf-bot/containerize/ ---- - -## Prerequisites - -> [!NOTE] -> -> GenAI applications can often benefit from GPU acceleration. Currently Docker Desktop supports GPU acceleration only on [Windows with the WSL2 backend](/manuals/desktop/features/gpu.md#using-nvidia-gpus-with-wsl2). Linux users can also access GPU acceleration using a native installation of the [Docker Engine](/manuals/engine/install/_index.md). - -- You have installed the latest version of [Docker Desktop](/get-started/get-docker.md) or, if you are a Linux user and are planning to use GPU acceleration, [Docker Engine](/manuals/engine/install/_index.md). Docker adds new features regularly and some parts of this guide may work only with the latest version of Docker Desktop. -- You have a [git client](https://git-scm.com/downloads). The examples in this section use a command-line based git client, but you can use any client. - -## Overview - -This section walks you through containerizing a generative AI (GenAI) application using Docker Desktop. - -> [!NOTE] -> -> You can see more samples of containerized GenAI applications in the [GenAI Stack](https://github.com/docker/genai-stack) demo applications. - -## Get the sample application - -The sample application used in this guide is a modified version of the PDF Reader application from the [GenAI Stack](https://github.com/docker/genai-stack) demo applications. The application is a full stack Python application that lets you ask questions about a PDF file. - -The application uses [LangChain](https://www.langchain.com/) for orchestration, [Streamlit](https://streamlit.io/) for the UI, [Ollama](https://ollama.ai/) to run the LLM, and [Neo4j](https://neo4j.com/) to store vectors. - -Clone the sample application. Open a terminal, change directory to a directory that you want to work in, and run the following command to clone the repository: - -```console -$ git clone https://github.com/craig-osterhout/docker-genai-sample -``` - -You should now have the following files in your `docker-genai-sample` directory. - -```text -├── docker-genai-sample/ -│ ├── .gitignore -│ ├── app.py -│ ├── chains.py -│ ├── env.example -│ ├── requirements.txt -│ ├── util.py -│ ├── LICENSE -│ └── README.md -``` - -## Create Docker assets - -Now that you have an application, you can create the necessary Docker assets to -containerize it. - -> [!TIP] -> -> [Gordon](/ai/gordon/), Docker's AI assistant, can generate Docker assets for your project. Ask Gordon to create a Dockerfile, Compose file, and `.dockerignore` tailored to your application. - -Create the following files in your `docker-genai-sample` directory. - -```dockerfile {collapse=true,title=Dockerfile} -# syntax=docker/dockerfile:1 - -# Comments are provided throughout this file to help you get started. -# If you need more help, visit the Dockerfile reference guide at -# https://docs.docker.com/go/dockerfile-reference/ - -ARG PYTHON_VERSION=3.11.4 -FROM python:${PYTHON_VERSION}-slim as base - -# Prevents Python from writing pyc files. -ENV PYTHONDONTWRITEBYTECODE=1 - -# Keeps Python from buffering stdout and stderr to avoid situations where -# the application crashes without emitting any logs due to buffering. -ENV PYTHONUNBUFFERED=1 - -WORKDIR /app - -# Create a non-privileged user that the app will run under. -# See https://docs.docker.com/go/dockerfile-user-best-practices/ -ARG UID=10001 -RUN adduser \ - --disabled-password \ - --gecos "" \ - --home "/nonexistent" \ - --shell "/sbin/nologin" \ - --no-create-home \ - --uid "${UID}" \ - appuser - -# Download dependencies as a separate step to take advantage of Docker's caching. -# Leverage a cache mount to /root/.cache/pip to speed up subsequent builds. -# Leverage a bind mount to requirements.txt to avoid having to copy them into -# into this layer. -RUN --mount=type=cache,target=/root/.cache/pip \ - --mount=type=bind,source=requirements.txt,target=requirements.txt \ - python -m pip install -r requirements.txt - -# Switch to the non-privileged user to run the application. -USER appuser - -# Copy the source code into the container. -COPY . . - -# Expose the port that the application listens on. -EXPOSE 8000 - -# Run the application. -CMD ["streamlit", "run", "app.py", "--server.address=0.0.0.0", "--server.port=8000"] -``` - -```yaml {collapse=true,title=compose.yaml} -# Comments are provided throughout this file to help you get started. -# If you need more help, visit the Docker Compose reference guide at -# https://docs.docker.com/go/compose-spec-reference/ - -# Here the instructions define your application as a service called "server". -# This service is built from the Dockerfile in the current directory. -# You can add other services your application may depend on here, such as a -# database or a cache. For examples, see the Awesome Compose repository: -# https://github.com/docker/awesome-compose -services: - server: - build: - context: . - ports: - - 8000:8000 - -# The commented out section below is an example of how to define a PostgreSQL -# database that your application can use. `depends_on` tells Docker Compose to -# start the database before your application. The `db-data` volume persists the -# database data between container restarts. The `db-password` secret is used -# to set the database password. You must create `db/password.txt` and add -# a password of your choosing to it before running `docker compose up`. -# depends_on: -# db: -# condition: service_healthy -# db: -# image: postgres -# restart: always -# user: postgres -# secrets: -# - db-password -# volumes: -# - db-data:/var/lib/postgresql/data -# environment: -# - POSTGRES_DB=example -# - POSTGRES_PASSWORD_FILE=/run/secrets/db-password -# expose: -# - 5432 -# healthcheck: -# test: [ "CMD", "pg_isready" ] -# interval: 10s -# timeout: 5s -# retries: 5 -# volumes: -# db-data: -# secrets: -# db-password: -# file: db/password.txt -``` - -```text {collapse=true,title=".dockerignore"} -# Include any files or directories that you don't want to be copied to your -# container here (e.g., local build artifacts, temporary files, etc.). -# -# For more help, visit the .dockerignore file reference guide at -# https://docs.docker.com/go/build-context-dockerignore/ - -**/.DS_Store -**/__pycache__ -**/.venv -**/.classpath -**/.dockerignore -**/.env -**/.git -**/.gitignore -**/.project -**/.settings -**/.toolstarget -**/.vs -**/.vscode -**/*.*proj.user -**/*.dbmdl -**/*.jfm -**/bin -**/charts -**/docker-compose* -**/compose.y*ml -**/Dockerfile* -**/node_modules -**/npm-debug.log -**/obj -**/secrets.dev.yaml -**/values.dev.yaml -LICENSE -README.md -``` - -You should now have the following contents in your `docker-genai-sample` -directory. - -```text -├── docker-genai-sample/ -│ ├── .dockerignore -│ ├── .gitignore -│ ├── app.py -│ ├── chains.py -│ ├── compose.yaml -│ ├── env.example -│ ├── requirements.txt -│ ├── util.py -│ ├── Dockerfile -│ ├── LICENSE -│ └── README.md -``` - -To learn more about these files, see the following: - -- [Dockerfile](../../../reference/dockerfile.md) -- [.dockerignore](../../../reference/dockerfile.md#dockerignore-file) -- [compose.yaml](/reference/compose-file/_index.md) - -## Run the application - -Inside the `docker-genai-sample` directory, run the following command in a -terminal. - -```console -$ docker compose up --build -``` - -Docker builds and runs your application. Depending on your network connection, it may take several minutes to download all the dependencies. You'll see a message like the following in the terminal when the application is running. - -```console -server-1 | You can now view your Streamlit app in your browser. -server-1 | -server-1 | URL: http://0.0.0.0:8000 -server-1 | -``` - -Open a browser and view the application at [http://localhost:8000](http://localhost:8000). You should see a simple Streamlit application. The application may take a few minutes to download the embedding model. While the download is in progress, **Running** appears in the top-right corner. - -The application requires a Neo4j database service and an LLM service to -function. If you have access to services that you ran outside of Docker, specify -the connection information and try it out. If you don't have the services -running, continue with this guide to learn how you can run some or all of these -services with Docker. - -In the terminal, press `ctrl`+`c` to stop the application. - -## Summary - -In this section, you learned how you can containerize and run your GenAI -application using Docker. - -## Next steps - -In the next section, you'll learn how you can run your application, database, and LLM service all locally using Docker. diff --git a/content/guides/genai-pdf-bot/develop.md b/content/guides/genai-pdf-bot/develop.md deleted file mode 100644 index 7a0dcae854ed..000000000000 --- a/content/guides/genai-pdf-bot/develop.md +++ /dev/null @@ -1,261 +0,0 @@ ---- -title: Use containers for generative AI development -linkTitle: Develop your app -weight: 20 -keywords: python, local, development, generative ai, genai, llm, neo4j, ollama, langchain, openai -description: Learn how to develop your generative AI (GenAI) application locally. -aliases: - - /guides/use-case/genai-pdf-bot/develop/ ---- - -## Prerequisites - -Complete [Containerize a generative AI application](containerize.md). - -## Overview - -In this section, you'll learn how to set up a development environment to access all the services that your generative AI (GenAI) application needs. This includes: - -- Adding a local database -- Adding a local or remote LLM service - -> [!NOTE] -> -> You can see more samples of containerized GenAI applications in the [GenAI Stack](https://github.com/docker/genai-stack) demo applications. - -## Add a local database - -You can use containers to set up local services, like a database. In this section, you'll update the `compose.yaml` file to define a database service. In addition, you'll specify an environment variables file to load the database connection information rather than manually entering the information every time. - -To run the database service: - -1. In the cloned repository's directory, rename `env.example` file to `.env`. - This file contains the environment variables that the containers will use. -2. In the cloned repository's directory, open the `compose.yaml` file in an IDE or text editor. -3. In the `compose.yaml` file, add the following: - - - Add instructions to run a Neo4j database - - Specify the environment file under the server service in order to pass in the environment variables for the connection - - The following is the updated `compose.yaml` file. All comments have been removed. - - ```yaml{hl_lines=["7-23"]} - services: - server: - build: - context: . - ports: - - 8000:8000 - env_file: - - .env - depends_on: - database: - condition: service_healthy - database: - image: neo4j:5.11 - ports: - - "7474:7474" - - "7687:7687" - environment: - - NEO4J_AUTH=${NEO4J_USERNAME}/${NEO4J_PASSWORD} - healthcheck: - test: ["CMD-SHELL", "wget --no-verbose --tries=1 --spider localhost:7474 || exit 1"] - interval: 5s - timeout: 3s - retries: 5 - ``` - - > [!NOTE] - > - > To learn more about Neo4j, see the [Neo4j Official Docker Image](https://hub.docker.com/_/neo4j). - -4. Run the application. Inside the `docker-genai-sample` directory, - run the following command in a terminal. - - ```console - $ docker compose up --build - ``` - -5. Access the application. Open a browser and view the application at [http://localhost:8000](http://localhost:8000). You should see a simple Streamlit application. Note that asking questions to a PDF will cause the application to fail because the LLM service specified in the `.env` file isn't running yet. - -6. Stop the application. In the terminal, press `ctrl`+`c` to stop the application. - -## Add a local or remote LLM service - -The sample application supports both [Ollama](https://ollama.ai/) and [OpenAI](https://openai.com/). This guide provides instructions for the following scenarios: - -- Run Ollama in a container -- Run Ollama outside of a container -- Use OpenAI - -While all platforms can use any of the previous scenarios, the performance and -GPU support may vary. You can use the following guidelines to help you choose the appropriate option: - -- Run Ollama in a container if you're on Linux, and using a native installation of the Docker Engine, or Windows 10/11, and using Docker Desktop, you - have a CUDA-supported GPU, and your system has at least 8 GB of RAM. -- Run Ollama outside of a container if you're on an Apple silicon Mac. -- Use OpenAI if the previous two scenarios don't apply to you. - -Choose one of the following options for your LLM service. - -{{< tabs >}} -{{< tab name="Run Ollama in a container" >}} - -When running Ollama in a container, you should have a CUDA-supported GPU. While you can run Ollama in a container without a supported GPU, the performance may not be acceptable. Only Linux and Windows 11 support GPU access to containers. - -To run Ollama in a container and provide GPU access: - -1. Install the prerequisites. - - For Docker Engine on Linux, install the [NVIDIA Container Toolkit](https://github.com/NVIDIA/nvidia-container-toolkit). - - For Docker Desktop on Windows 10/11, install the latest [NVIDIA driver](https://www.nvidia.com/Download/index.aspx) and make sure you are using the [WSL2 backend](/manuals/desktop/features/wsl/_index.md#turn-on-docker-desktop-wsl-2) -2. Add the Ollama service and a volume in your `compose.yaml`. The following is - the updated `compose.yaml`: - - ```yaml {hl_lines=["24-38"]} - services: - server: - build: - context: . - ports: - - 8000:8000 - env_file: - - .env - depends_on: - database: - condition: service_healthy - database: - image: neo4j:5.11 - ports: - - "7474:7474" - - "7687:7687" - environment: - - NEO4J_AUTH=${NEO4J_USERNAME}/${NEO4J_PASSWORD} - healthcheck: - test: - [ - "CMD-SHELL", - "wget --no-verbose --tries=1 --spider localhost:7474 || exit 1", - ] - interval: 5s - timeout: 3s - retries: 5 - ollama: - image: ollama/ollama:latest - ports: - - "11434:11434" - volumes: - - ollama_volume:/root/.ollama - deploy: - resources: - reservations: - devices: - - driver: nvidia - count: all - capabilities: [gpu] - volumes: - ollama_volume: - ``` - - > [!NOTE] - > - > For more details about the Compose instructions, see [Turn on GPU access with Docker Compose](/manuals/compose/how-tos/gpu-support.md). - -3. Add the ollama-pull service to your `compose.yaml` file. This service uses - the `docker/genai:ollama-pull` image, based on the GenAI Stack's - [pull_model.Dockerfile](https://github.com/docker/genai-stack/blob/main/pull_model.Dockerfile). - The service will automatically pull the model for your Ollama - container. The following is the updated section of the `compose.yaml` file: - - ```yaml {hl_lines=["12-17"]} - services: - server: - build: - context: . - ports: - - 8000:8000 - env_file: - - .env - depends_on: - database: - condition: service_healthy - ollama-pull: - condition: service_completed_successfully - ollama-pull: - image: docker/genai:ollama-pull - env_file: - - .env - # ... - ``` - -{{< /tab >}} -{{< tab name="Run Ollama outside of a container" >}} - -To run Ollama outside of a container: - -1. [Install](https://github.com/jmorganca/ollama) and run Ollama on your host - machine. -2. Update the `OLLAMA_BASE_URL` value in your `.env` file to - `http://host.docker.internal:11434`. -3. Pull the model to Ollama using the following command. - ```console - $ ollama pull llama2 - ``` - -{{< /tab >}} -{{< tab name="Use OpenAI" >}} - -> [!IMPORTANT] -> -> Using OpenAI requires an [OpenAI account](https://platform.openai.com/login). OpenAI is a third-party hosted service and charges may apply. - -1. Update the `LLM` value in your `.env` file to - `gpt-3.5`. -2. Uncomment and update the `OPENAI_API_KEY` value in your `.env` file to - your [OpenAI API key](https://help.openai.com/en/articles/4936850-where-do-i-find-my-api-key). - -{{< /tab >}} -{{< /tabs >}} - -## Run your GenAI application - -At this point, you have the following services in your Compose file: - -- Server service for your main GenAI application -- Database service to store vectors in a Neo4j database -- (optional) Ollama service to run the LLM -- (optional) Ollama-pull service to automatically pull the model for the Ollama - service - -To run all the services, run the following command in your `docker-genai-sample` -directory: - -```console -$ docker compose up --build -``` - -If your Compose file has the ollama-pull service, it may take several minutes for the ollama-pull service to pull the model. The ollama-pull service will continuously update the console with its status. After pulling the model, the ollama-pull service container will stop and you can access the application. - -Once the application is running, open a browser and access the application at [http://localhost:8000](http://localhost:8000). - -Upload a PDF file, for example the [Docker CLI Cheat Sheet](https://docs.docker.com/get-started/docker_cheatsheet.pdf), and ask a question about the PDF. - -Depending on your system and the LLM service that you chose, it may take several -minutes to answer. If you are using Ollama and the performance isn't -acceptable, try using OpenAI. - -## Summary - -In this section, you learned how to set up a development environment to provide -access all the services that your GenAI application needs. - -Related information: - -- [Dockerfile reference](../../../reference/dockerfile.md) -- [Compose file reference](/reference/compose-file/_index.md) -- [Ollama Docker image](https://hub.docker.com/r/ollama/ollama) -- [Neo4j Official Docker Image](https://hub.docker.com/_/neo4j) -- [GenAI Stack demo applications](https://github.com/docker/genai-stack) - -## Next steps - -See samples of more GenAI applications in the [GenAI Stack demo applications](https://github.com/docker/genai-stack). diff --git a/content/guides/genai-video-bot/index.md b/content/guides/genai-video-bot/index.md index a5d5b6497aca..7a7dd5f30e0b 100644 --- a/content/guides/genai-video-bot/index.md +++ b/content/guides/genai-video-bot/index.md @@ -6,10 +6,10 @@ keywords: python, generative ai, genai, llm, whisper, pinecone, openai, whisper summary: | Learn how to build and deploy a generative AI video analysis and transcription bot using Docker. -tags: [ai] aliases: - /guides/use-case/genai-video-bot/ params: + tags: [ai] time: 20 minutes --- @@ -78,14 +78,18 @@ addition, it provides timestamps from the video that can help you find the sourc called `.env` and specify your API keys inside. The following is the contents of the `.env.example` file that you can refer to as an example. ```text - #---------------------------------------------------------------------------- + #------------------------------------------------------------------------- +--- # OpenAI - #---------------------------------------------------------------------------- + #------------------------------------------------------------------------- +--- OPENAI_TOKEN=your-api-key # Replace your-api-key with your personal API key - #---------------------------------------------------------------------------- + #------------------------------------------------------------------------- +--- # Pinecone - #---------------------------------------------------------------------------- + #------------------------------------------------------------------------- +--- PINECONE_TOKEN=your-api-key # Replace your-api-key with your personal API key ``` diff --git a/content/guides/gha.md b/content/guides/gha.md index ec61aeeacad7..4da6330e797f 100644 --- a/content/guides/gha.md +++ b/content/guides/gha.md @@ -5,7 +5,7 @@ summary: | Learn how to automate image build and push with GitHub Actions. keywords: github actions, ci/cd, docker hub, build and push, automation, workflows params: - tags: [devops] + tags: [cicd] time: 10 minutes --- @@ -252,5 +252,6 @@ additional features based on your project's needs, such as ### Further reading - Learn more about advanced configurations and examples in the [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) section. -- For more complex build setups, you may want to consider [Bake](/manuals/build/bake/_index.md). (See also the [Mastering Buildx Bake guide](/guides/bake/index.md).) +- For more complex build setups, you may want to consider [Bake](/manuals/build/bake/_index.md). (See also the [Mastering Buildx Bake guide](/guides/bake/).) + - Learn about Docker's managed build service, designed for faster, multi-platform builds, see [Docker Build Cloud](/guides/docker-build-cloud/_index.md). diff --git a/content/guides/github-sonarqube-sandbox/_index.md b/content/guides/github-sonarqube-sandbox/_index.md index 90d0684a2e05..f2dbb69bac43 100644 --- a/content/guides/github-sonarqube-sandbox/_index.md +++ b/content/guides/github-sonarqube-sandbox/_index.md @@ -4,19 +4,18 @@ linkTitle: AI-powered code quality summary: Build AI-powered code quality workflows using E2B sandboxes with Docker's MCP catalog to automate GitHub and SonarQube integration. description: Learn how to create E2B sandboxes with MCP servers, analyze code quality with SonarQube, and generate quality-gated pull requests using GitHub—all through natural language interactions with Claude. keywords: sonarqube, e2b, sandboxes, mcp, github, code quality, ai workflow, claude -tags: [devops] +aliases: + - /guides/github-sonarqube-sandbox/customize/ + - /guides/github-sonarqube-sandbox/troubleshoot/ + - /guides/github-sonarqube-sandbox/workflow/ params: + tags: [cicd] time: 40 minutes image: - resource_links: - - title: E2B Documentation - url: https://e2b.dev/docs - - title: Docker MCP Catalog - url: https://hub.docker.com/mcp - - title: Sandboxes url: https://docs.docker.com/ai/mcp-catalog-and-toolkit/sandboxes/ --- + This guide demonstrates how to build an AI-powered code quality workflow using [E2B sandboxes](https://e2b.dev/docs) with Docker’s MCP catalog. You’ll create a system that automatically analyzes code quality issues in GitHub repositories @@ -54,3 +53,2089 @@ cloud without consuming local resources ## Learn more Read Docker's blog post: [Docker + E2B: Building the Future of Trusted AI](https://www.docker.com/blog/docker-e2b-building-the-future-of-trusted-ai/). + +## Build a code quality check workflow + +In this section, you'll build a complete code quality automation workflow +step-by-step. You'll start by creating an E2B sandbox with GitHub and +SonarQube MCP servers, then progressively add functionality until you have a +production-ready workflow that analyzes code quality and creates pull requests. + +By working through each step sequentially, you'll learn how MCP servers work, +how to interact with them through Claude, and how to chain operations together +to build powerful automation workflows. + +### Prerequisites + +Before you begin, make sure you have: + +- E2B account with [API access](https://e2b.dev/docs/api-key) +- [Anthropic API key](https://docs.claude.com/en/api/admin-api/apikeys/get-api-key) + + > [!NOTE] + > + > This example uses Claude CLI which comes pre-installed in E2B sandboxes, but you can adapt the example to work with other AI assistants of your choice. See [E2B's MCP documentation](https://e2b.dev/docs/mcp/quickstart) for alternative connection methods. + +- GitHub account with: + - A repository containing code to analyze + - [Personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) with `repo` scope +- SonarCloud account with: + - [Organization](https://docs.sonarsource.com/sonarqube-cloud/administering-sonarcloud/resources-structure/organization) created + - [Project configured](https://docs.sonarsource.com/sonarqube-community-build/project-administration/creating-and-importing-projects) for your repository + - [User token](https://docs.sonarsource.com/sonarqube-server/instance-administration/security/administering-tokens) generated +- Language runtime installed: + - TypeScript: [Node.js 18+](https://nodejs.org/en/download) + - Python: [Python 3.8+](https://www.python.org/downloads/) + +> [!NOTE] +> +> This guide uses Claude's `--dangerously-skip-permissions` flag to enable +> automated command execution in E2B sandboxes. This flag bypasses permission +> prompts, which is appropriate for isolated container environments like E2B +> where sandboxes are disposable and separate from your local machine. +> +> However, be aware that Claude can execute any commands within the sandbox, +> including accessing files and credentials available in that environment. Only +> use this approach with trusted code and workflows. For more information, +> see [Anthropic's guidance on container security](https://docs.anthropic.com/en/docs/claude-code/devcontainer). + +### Set up your project + +{{< tabs group="language" >}} +{{< tab name="TypeScript" >}} + +1. Create a new directory for your workflow and initialize Node.js: + + ```bash + mkdir github-sonarqube-workflow + cd github-sonarqube-workflow + npm init -y + ``` + +2. Open `package.json` and configure it for ES modules: + + ```json + { + "name": "github-sonarqube-workflow", + "version": "1.0.0", + "description": "Automated code quality workflow using E2B, GitHub, and SonarQube", + "type": "module", + "main": "quality-workflow.ts", + "scripts": { + "start": "tsx quality-workflow.ts" + }, + "keywords": ["e2b", "github", "sonarqube", "mcp", "code-quality"], + "author": "", + "license": "MIT" + } + ``` + +3. Install required dependencies: + + ```bash + npm install e2b dotenv + npm install -D typescript tsx @types/node + ``` + +4. Create a `.env` file in your project root: + + ```bash + touch .env + ``` + +5. Add your API keys and configuration, replacing the placeholders with your actual credentials: + + ```plaintext + E2B_API_KEY=your_e2b_api_key_here + ANTHROPIC_API_KEY=your_anthropic_api_key_here + GITHUB_TOKEN=ghp_your_personal_access_token_here + GITHUB_OWNER=your_github_username + GITHUB_REPO=your_repository_name + SONARQUBE_ORG=your_sonarcloud_org_key + SONARQUBE_TOKEN=your_sonarqube_user_token + SONARQUBE_URL=https://sonarcloud.io + ``` + +6. Protect your credentials by adding `.env` to `.gitignore`: + + ```bash + echo ".env" >> .gitignore + echo "node_modules/" >> .gitignore + ``` + +{{< /tab >}} +{{< tab name="Python" >}} + +1. Create a new directory for your workflow: + + ```bash + mkdir github-sonarqube-workflow + cd github-sonarqube-workflow + ``` + +2. Create a virtual environment and activate it: + + ```bash + python3 -m venv venv + source venv/bin/activate # On Windows: venv\Scripts\activate + ``` + +3. Install required dependencies: + + ```bash + pip install e2b python-dotenv + ``` + +4. Create a `.env` file in your project root: + + ```bash + touch .env + ``` + +5. Add your API keys and configuration, replacing the placeholders with your actual credentials: + + ```plaintext + E2B_API_KEY=your_e2b_api_key_here + ANTHROPIC_API_KEY=your_anthropic_api_key_here + GITHUB_TOKEN=ghp_your_personal_access_token_here + GITHUB_OWNER=your_github_username + GITHUB_REPO=your_repository_name + SONARQUBE_ORG=your_sonarcloud_org_key + SONARQUBE_TOKEN=your_sonarqube_user_token + SONARQUBE_URL=https://sonarcloud.io + ``` + +6. Protect your credentials by adding `.env` to `.gitignore`: + + ```bash + echo ".env" >> .gitignore + echo "venv/" >> .gitignore + echo "__pycache__/" >> .gitignore + ``` + +{{< /tab >}} +{{< /tabs >}} + +### Step 1: Create your first sandbox + +Let's start by creating a sandbox and verifying the MCP servers are configured correctly. + +{{< tabs group="language" >}} +{{< tab name="TypeScript" >}} + +Create a file named `01-test-connection.ts` in your project root: + +```typescript +import "dotenv/config"; +import { Sandbox } from "e2b"; + +async function testConnection() { + console.log( + "Creating E2B sandbox with GitHub and SonarQube MCP servers...\n", + ); + + const sbx = await Sandbox.betaCreate({ + envs: { + ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY!, + GITHUB_TOKEN: process.env.GITHUB_TOKEN!, + SONARQUBE_TOKEN: process.env.SONARQUBE_TOKEN!, + }, + mcp: { + githubOfficial: { + githubPersonalAccessToken: process.env.GITHUB_TOKEN!, + }, + sonarqube: { + org: process.env.SONARQUBE_ORG!, + token: process.env.SONARQUBE_TOKEN!, + url: "https://sonarcloud.io", + }, + }, + }); + + const mcpUrl = sbx.betaGetMcpUrl(); + const mcpToken = await sbx.betaGetMcpToken(); + + console.log(" Sandbox created successfully!"); + console.log(`MCP Gateway URL: ${mcpUrl}\n`); + + // Wait for MCP initialization + await new Promise((resolve) => setTimeout(resolve, 1000)); + + // Configure Claude to use the MCP gateway + console.log("Connecting Claude CLI to MCP gateway..."); + await sbx.commands.run( + `claude mcp add --transport http e2b-mcp-gateway ${mcpUrl} --header "Authorization: Bearer ${mcpToken}"`, + { + timeoutMs: 0, + onStdout: console.log, + onStderr: console.log, + }, + ); + + console.log("\nœConnection successful! Cleaning up..."); + await sbx.kill(); +} + +testConnection().catch(console.error); +``` + +Run this script to verify your setup: + +```bash +npx tsx 01-test-connection.ts +``` + +{{< /tab >}} +{{< tab name="Python" >}} + +Create a file named `01_test_connection.py` in your project root: + +```python +import os +import asyncio +from dotenv import load_dotenv +from e2b import AsyncSandbox + +load_dotenv() + +async def test_connection(): + print("Creating E2B sandbox with GitHub and SonarQube MCP servers...\n") + + sbx = await AsyncSandbox.beta_create( + envs={ + "ANTHROPIC_API_KEY": os.getenv("ANTHROPIC_API_KEY"), + "GITHUB_TOKEN": os.getenv("GITHUB_TOKEN"), + "SONARQUBE_TOKEN": os.getenv("SONARQUBE_TOKEN"), + }, + mcp={ + "githubOfficial": { + "githubPersonalAccessToken": os.getenv("GITHUB_TOKEN"), + }, + "sonarqube": { + "org": os.getenv("SONARQUBE_ORG"), + "token": os.getenv("SONARQUBE_TOKEN"), + "url": "https://sonarcloud.io", + }, + }, + ) + + mcp_url = sbx.beta_get_mcp_url() + mcp_token = await sbx.beta_get_mcp_token() + + print(" Sandbox created successfully!") + print(f"MCP Gateway URL: {mcp_url}\n") + + # Wait for MCP initialization + await asyncio.sleep(1) + + # Configure Claude to use the MCP gateway + print("Connecting Claude CLI to MCP gateway...") + await sbx.commands.run( + f'claude mcp add --transport http e2b-mcp-gateway {mcp_url} --header "Authorization: Bearer {mcp_token}"', + timeout=0, + on_stdout=print, + on_stderr=print, + ) + + print("\n Connection successful! Cleaning up...") + await sbx.kill() + +if __name__ == "__main__": + asyncio.run(test_connection()) +``` + +Run this script to verify your setup: + +```bash +python 01_test_connection.py +``` + +{{< /tab >}} +{{< /tabs >}} + +Your output should look similar to the following example: + +```console {collapse=true} +Creating E2B sandbox with GitHub and SonarQube MCP servers... + +✓ Sandbox created successfully! +MCP Gateway URL: https://50005-xxxxx.e2b.app/mcp + +Connecting Claude CLI to MCP gateway... +Added HTTP MCP server e2b-mcp-gateway with URL: https://50005-xxxxx.e2b.app/mcp to local config +Headers: { + "Authorization": "Bearer xxxxx-xxxx-xxxx" +} +File modified: /home/user/.claude.json [project: /home/user] + +✓ Connection successful! Cleaning up... +``` + +You've just learned how to create an E2B sandbox with multiple MCP servers +configured. The `betaCreate` method initializes a cloud environment +with Claude CLI and your specified MCP servers. + +### Step 2: Discover available MCP tools + +MCP servers expose tools that Claude can call. The GitHub MCP server provides +repository management tools, while SonarQube provides code analysis tools. +By listing their tools, you know what operations are possible. + +To try listing MCP tools: + +{{< tabs group="language" >}} +{{< tab name="TypeScript" >}} + +Create `02-list-tools.ts`: + +```typescript +import "dotenv/config"; +import { Sandbox } from "e2b"; + +async function listTools() { + console.log("Creating sandbox...\n"); + + const sbx = await Sandbox.betaCreate({ + envs: { + ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY!, + GITHUB_TOKEN: process.env.GITHUB_TOKEN!, + SONARQUBE_TOKEN: process.env.SONARQUBE_TOKEN!, + }, + mcp: { + githubOfficial: { + githubPersonalAccessToken: process.env.GITHUB_TOKEN!, + }, + sonarqube: { + org: process.env.SONARQUBE_ORG!, + token: process.env.SONARQUBE_TOKEN!, + url: "https://sonarcloud.io", + }, + }, + }); + + const mcpUrl = sbx.betaGetMcpUrl(); + const mcpToken = await sbx.betaGetMcpToken(); + + // Wait for MCP initialization + await new Promise((resolve) => setTimeout(resolve, 1000)); + + await sbx.commands.run( + `claude mcp add --transport http e2b-mcp-gateway ${mcpUrl} --header "Authorization: Bearer ${mcpToken}"`, + { timeoutMs: 0, onStdout: console.log, onStderr: console.log }, + ); + + console.log("\nDiscovering available MCP tools...\n"); + + const prompt = + "List all MCP tools you have access to. For each tool, show its exact name and a brief description."; + + await sbx.commands.run( + `echo '${prompt}' | claude -p --dangerously-skip-permissions`, + { timeoutMs: 0, onStdout: console.log, onStderr: console.log }, + ); + + await sbx.kill(); +} + +listTools().catch(console.error); +``` + +Run the script: + +```bash +npx tsx 02-list-tools.ts +``` + +{{< /tab >}} +{{< tab name="Python" >}} + +Create `02_list_tools.py`: + +```python +import os +import asyncio +from dotenv import load_dotenv +from e2b import AsyncSandbox + +load_dotenv() + +async def list_tools(): + print("Creating sandbox...\n") + + sbx = await AsyncSandbox.beta_create( + envs={ + "ANTHROPIC_API_KEY": os.getenv("ANTHROPIC_API_KEY"), + "GITHUB_TOKEN": os.getenv("GITHUB_TOKEN"), + "SONARQUBE_TOKEN": os.getenv("SONARQUBE_TOKEN"), + }, + mcp={ + "githubOfficial": { + "githubPersonalAccessToken": os.getenv("GITHUB_TOKEN"), + }, + "sonarqube": { + "org": os.getenv("SONARQUBE_ORG"), + "token": os.getenv("SONARQUBE_TOKEN"), + "url": "https://sonarcloud.io", + }, + }, + ) + + mcp_url = sbx.beta_get_mcp_url() + mcp_token = await sbx.beta_get_mcp_token() + + # Wait for MCP initialization + await asyncio.sleep(1) + + await sbx.commands.run( + f'claude mcp add --transport http e2b-mcp-gateway {mcp_url} --header "Authorization: Bearer {mcp_token}"', + timeout=0, + on_stdout=print, + on_stderr=print, + ) + + print("\nDiscovering available MCP tools...\n") + + prompt = "List all MCP tools you have access to. For each tool, show its exact name and a brief description." + + await sbx.commands.run( + f"echo '{prompt}' | claude -p --dangerously-skip-permissions", + timeout=0, + on_stdout=print, + on_stderr=print, + ) + + await sbx.kill() + +if __name__ == "__main__": + asyncio.run(list_tools()) +``` + +Run the script: + +```bash +python 02_list_tools.py +``` + +{{< /tab >}} +{{< /tabs >}} + +In the console, you should see a list of MCP tools: + +```console {collapse=true} +Creating sandbox... + +Sandbox created +Connecting to MCP gateway... + +Discovering available MCP tools... + +I have access to the following MCP tools: + +**GitHub Tools:** +1. mcp__create_repository - Create a new GitHub repository +2. mcp__list_issues - List issues in a repository +3. mcp__create_issue - Create a new issue +4. mcp__get_file_contents - Get file contents from a repository +5. mcp__create_or_update_file - Create or update files in a repository +6. mcp__create_pull_request - Create a pull request +7. mcp__create_branch - Create a new branch +8. mcp__push_files - Push multiple files in a single commit +... (30+ more GitHub tools) + +**SonarQube Tools:** +1. mcp__get_projects - List projects in organization +2. mcp__get_quality_gate_status - Get quality gate status for a project +3. mcp__list_project_issues - List quality issues in a project +4. mcp__search_issues - Search for specific quality issues +... (SonarQube analysis tools) +``` + +### Step 3: Test GitHub MCP tools + +Let's try testing GitHub using MCP tools. Start simple by listing +repository issues. + +{{< tabs group="language" >}} +{{< tab name="TypeScript" >}} + +Create `03-test-github.ts`: + +```typescript +import "dotenv/config"; +import { Sandbox } from "e2b"; + +async function testGitHub() { + console.log("Creating sandbox...\n"); + + const sbx = await Sandbox.betaCreate({ + envs: { + ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY!, + GITHUB_TOKEN: process.env.GITHUB_TOKEN!, + }, + mcp: { + githubOfficial: { + githubPersonalAccessToken: process.env.GITHUB_TOKEN!, + }, + }, + }); + + const mcpUrl = sbx.betaGetMcpUrl(); + const mcpToken = await sbx.betaGetMcpToken(); + + await new Promise((resolve) => setTimeout(resolve, 1000)); + + await sbx.commands.run( + `claude mcp add --transport http e2b-mcp-gateway ${mcpUrl} --header "Authorization: Bearer ${mcpToken}"`, + { timeoutMs: 0, onStdout: console.log, onStderr: console.log }, + ); + + const repoPath = `${process.env.GITHUB_OWNER}/${process.env.GITHUB_REPO}`; + + console.log(`\nListing issues in ${repoPath}...\n`); + + const prompt = `Using the GitHub MCP tools, list all open issues in the repository "${repoPath}". Show the issue number, title, and author for each.`; + + await sbx.commands.run( + `echo '${prompt.replace(/'/g, "'\\''")}' | claude -p --dangerously-skip-permissions`, + { + timeoutMs: 0, + onStdout: console.log, + onStderr: console.log, + }, + ); + + await sbx.kill(); +} + +testGitHub().catch(console.error); +``` + +Run the script: + +```bash +npx tsx 03-test-github.ts +``` + +{{< /tab >}} +{{< tab name="Python" >}} + +Create `03_test_github.py`: + +```python +import os +import asyncio +from dotenv import load_dotenv +from e2b import AsyncSandbox + +load_dotenv() + +async def test_github(): + print("Creating sandbox...\n") + + sbx = await AsyncSandbox.beta_create( + envs={ + "ANTHROPIC_API_KEY": os.getenv("ANTHROPIC_API_KEY"), + "GITHUB_TOKEN": os.getenv("GITHUB_TOKEN"), + }, + mcp={ + "githubOfficial": { + "githubPersonalAccessToken": os.getenv("GITHUB_TOKEN"), + }, + }, + ) + + mcp_url = sbx.beta_get_mcp_url() + mcp_token = await sbx.beta_get_mcp_token() + + await asyncio.sleep(1) + + await sbx.commands.run( + f'claude mcp add --transport http e2b-mcp-gateway {mcp_url} --header "Authorization: Bearer {mcp_token}"', + timeout=0, + on_stdout=print, + on_stderr=print, + ) + + repo_path = f"{os.getenv('GITHUB_OWNER')}/{os.getenv('GITHUB_REPO')}" + + print(f"\nListing issues in {repo_path}...\n") + + prompt = f'Using the GitHub MCP tools, list all open issues in the repository "{repo_path}". Show the issue number, title, and author for each.' + + await sbx.commands.run( + f"echo '{prompt}' | claude -p --dangerously-skip-permissions", + timeout=0, + on_stdout=print, + on_stderr=print, + ) + + await sbx.kill() + +if __name__ == "__main__": + asyncio.run(test_github()) +``` + +Run the script: + +```bash +python 03_test_github.py +``` + +{{< /tab >}} +{{< /tabs >}} + +You should see Claude use the GitHub MCP tools to list your repository's issues: + +```console {collapse=true} +Creating sandbox... +Connecting to MCP gateway... + +Listing issues in ... + +Here are the first 10 open issues in the repository: + +1. **Issue #23577**: Update README (author: user1) +2. **Issue #23575**: release-notes for Compose v2.40.1 version (author: user2) +3. **Issue #23570**: engine-cli: fix `docker volume prune` output (author: user3) +4. **Issue #23568**: Engdocs update (author: user4) +5. **Issue #23565**: add new section (author: user5) +... (continues with more issues) +``` + +You can now send prompts to Claude and interact with GitHub through +natural language. Claude decides what tool to call based on your prompt. + +### Step 4: Test SonarQube MCP tools + +Let's analyze code quality using SonarQube MCP tools. + +{{< tabs group="language" >}} +{{< tab name="TypeScript" >}} + +Create `04-test-sonarqube.ts`: + +```typescript +import "dotenv/config"; +import { Sandbox } from "e2b"; + +async function testSonarQube() { + console.log("Creating sandbox...\n"); + + const sbx = await Sandbox.betaCreate({ + envs: { + ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY!, + GITHUB_TOKEN: process.env.GITHUB_TOKEN!, + SONARQUBE_TOKEN: process.env.SONARQUBE_TOKEN!, + }, + mcp: { + githubOfficial: { + githubPersonalAccessToken: process.env.GITHUB_TOKEN!, + }, + sonarqube: { + org: process.env.SONARQUBE_ORG!, + token: process.env.SONARQUBE_TOKEN!, + url: "https://sonarcloud.io", + }, + }, + }); + + const mcpUrl = sbx.betaGetMcpUrl(); + const mcpToken = await sbx.betaGetMcpToken(); + + await new Promise((resolve) => setTimeout(resolve, 1000)); + + await sbx.commands.run( + `claude mcp add --transport http e2b-mcp-gateway ${mcpUrl} --header "Authorization: Bearer ${mcpToken}"`, + { timeoutMs: 0, onStdout: console.log, onStderr: console.log }, + ); + + console.log("\nAnalyzing code quality with SonarQube...\n"); + + const prompt = `Using the SonarQube MCP tools: + 1. List all projects in my organization + 2. For the first project, show: + - Quality gate status (pass/fail) + - Number of bugs + - Number of code smells + - Number of security vulnerabilities + 3. List the top 5 most critical issues found`; + + await sbx.commands.run( + `echo '${prompt.replace(/'/g, "'\\''")}' | claude -p --dangerously-skip-permissions`, + { + timeoutMs: 0, + onStdout: console.log, + onStderr: console.log, + }, + ); + + await sbx.kill(); +} + +testSonarQube().catch(console.error); +``` + +Run the script: + +```bash +npx tsx 04-test-sonarqube.ts +``` + +{{< /tab >}} +{{< tab name="Python" >}} + +Create `04_test_sonarqube.py`: + +```python +import os +import asyncio +from dotenv import load_dotenv +from e2b import AsyncSandbox + +load_dotenv() + +async def test_sonarqube(): + print("Creating sandbox...\n") + + sbx = await AsyncSandbox.beta_create( + envs={ + "ANTHROPIC_API_KEY": os.getenv("ANTHROPIC_API_KEY"), + "GITHUB_TOKEN": os.getenv("GITHUB_TOKEN"), + "SONARQUBE_TOKEN": os.getenv("SONARQUBE_TOKEN"), + }, + mcp={ + "githubOfficial": { + "githubPersonalAccessToken": os.getenv("GITHUB_TOKEN"), + }, + "sonarqube": { + "org": os.getenv("SONARQUBE_ORG"), + "token": os.getenv("SONARQUBE_TOKEN"), + "url": "https://sonarcloud.io", + }, + }, + ) + + mcp_url = sbx.beta_get_mcp_url() + mcp_token = await sbx.beta_get_mcp_token() + + await asyncio.sleep(1) + + await sbx.commands.run( + f'claude mcp add --transport http e2b-mcp-gateway {mcp_url} --header "Authorization: Bearer {mcp_token}"', + timeout=0, + on_stdout=print, + on_stderr=print, + ) + + print("\nAnalyzing code quality with SonarQube...\n") + + prompt = """Using the SonarQube MCP tools: + 1. List all projects in my organization + 2. For the first project, show: + - Quality gate status (pass/fail) + - Number of bugs + - Number of code smells + - Number of security vulnerabilities + 3. List the top 5 most critical issues found""" + + await sbx.commands.run( + f"echo '{prompt}' | claude -p --dangerously-skip-permissions", + timeout=0, + on_stdout=print, + on_stderr=print, + ) + + await sbx.kill() + +if __name__ == "__main__": + asyncio.run(test_sonarqube()) +``` + +Run the script: + +```bash +python 04_test_sonarqube.py +``` + +{{< /tab >}} +{{< /tabs >}} + +> [!NOTE] +> +> This script may take a few minutes to run. + +You should see Claude output SonarQube analysis results: + +```console {collapse=true} +Creating sandbox... + +Analyzing code quality with SonarQube... + +## SonarQube Analysis Results + +### 1. Projects in Your Organization + +Found **1 project**: +- **Project Name**: project-1 +- **Project Key**: project-testing + +### 2. Project Analysis + +... + +### 3. Top 5 Most Critical Issues + +Found 1 total issues (all are code smells with no critical/blocker severity): + +1. **MAJOR Severity** - test.js:2 + - **Rule**: javascript:S1854 + - **Message**: Remove this useless assignment to variable "unusedVariable" + - **Status**: OPEN + +**Summary**: The project is in good health with no bugs or vulnerabilities detected. +``` + +You can now use SonarQube MCP tools to analyze code quality through +natural language. You can retrieve quality metrics, identify issues, +and understand what code needs fixing. + +### Step 5: Create a branch and make code changes + +Now, let's teach Claude to fix code based on quality issues discovered +by SonarQube. + +{{< tabs group="language" >}} +{{< tab name="TypeScript" >}} + +Create `05-fix-code-issue.ts`: + +```typescript +import "dotenv/config"; +import { Sandbox } from "e2b"; + +async function fixCodeIssue() { + console.log("Creating sandbox...\n"); + + const sbx = await Sandbox.betaCreate({ + envs: { + ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY!, + GITHUB_TOKEN: process.env.GITHUB_TOKEN!, + SONARQUBE_TOKEN: process.env.SONARQUBE_TOKEN!, + }, + mcp: { + githubOfficial: { + githubPersonalAccessToken: process.env.GITHUB_TOKEN!, + }, + sonarqube: { + org: process.env.SONARQUBE_ORG!, + token: process.env.SONARQUBE_TOKEN!, + url: "https://sonarcloud.io", + }, + }, + }); + + const mcpUrl = sbx.betaGetMcpUrl(); + const mcpToken = await sbx.betaGetMcpToken(); + + await new Promise((resolve) => setTimeout(resolve, 1000)); + + await sbx.commands.run( + `claude mcp add --transport http e2b-mcp-gateway ${mcpUrl} --header "Authorization: Bearer ${mcpToken}"`, + { timeoutMs: 0, onStdout: console.log, onStderr: console.log }, + ); + + const repoPath = `${process.env.GITHUB_OWNER}/${process.env.GITHUB_REPO}`; + const branchName = `quality-fix-${Date.now()}`; + + console.log("\nFixing a code quality issue...\n"); + + const prompt = `Using GitHub and SonarQube MCP tools: + + 1. Analyze code quality in repository "${repoPath}" with SonarQube + 2. Find ONE simple issue that can be confidently fixed (like an unused variable or code smell) + 3. Create a new branch called "${branchName}" + 4. Read the file containing the issue using GitHub tools + 5. Fix the issue in the code + 6. Commit the fix to the new branch with a clear commit message + + Important: Only fix issues you're 100% confident about. Explain what you're fixing and why.`; + + await sbx.commands.run( + `echo '${prompt.replace(/'/g, "'\\''")}' | claude -p --dangerously-skip-permissions`, + { + timeoutMs: 0, + onStdout: console.log, + onStderr: console.log, + }, + ); + + console.log(`\nœCheck your repository for branch: ${branchName}`); + + await sbx.kill(); +} + +fixCodeIssue().catch(console.error); +``` + +Run the script: + +```bash +npx tsx 05-fix-code-issue.ts +``` + +{{< /tab >}} +{{< tab name="Python" >}} + +Create `05_fix_code_issue.py`: + +```python +import os +import asyncio +import time +from dotenv import load_dotenv +from e2b import AsyncSandbox + +load_dotenv() + +async def fix_code_issue(): + print("Creating sandbox...\n") + + sbx = await AsyncSandbox.beta_create( + envs={ + "ANTHROPIC_API_KEY": os.getenv("ANTHROPIC_API_KEY"), + "GITHUB_TOKEN": os.getenv("GITHUB_TOKEN"), + "SONARQUBE_TOKEN": os.getenv("SONARQUBE_TOKEN"), + }, + mcp={ + "githubOfficial": { + "githubPersonalAccessToken": os.getenv("GITHUB_TOKEN"), + }, + "sonarqube": { + "org": os.getenv("SONARQUBE_ORG"), + "token": os.getenv("SONARQUBE_TOKEN"), + "url": "https://sonarcloud.io", + }, + }, + ) + + mcp_url = sbx.beta_get_mcp_url() + mcp_token = await sbx.beta_get_mcp_token() + + await asyncio.sleep(1) + + await sbx.commands.run( + f'claude mcp add --transport http e2b-mcp-gateway {mcp_url} --header "Authorization: Bearer {mcp_token}"', + timeout=0, + on_stdout=print, + on_stderr=print, + ) + + repo_path = f"{os.getenv('GITHUB_OWNER')}/{os.getenv('GITHUB_REPO')}" + branch_name = f"quality-fix-{int(time.time() * 1000)}" + + print("\nFixing a code quality issue...\n") + + prompt = f"""Using GitHub and SonarQube MCP tools: + + 1. Analyze code quality in repository "{repo_path}" with SonarQube + 2. Find ONE simple issue that can be confidently fixed (like an unused variable or code smell) + 3. Create a new branch called "{branch_name}" + 4. Read the file containing the issue using GitHub tools + 5. Fix the issue in the code + 6. Commit the fix to the new branch with a clear commit message + + Important: Only fix issues you're 100% confident about. Explain what you're fixing and why.""" + + await sbx.commands.run( + f"echo '{prompt}' | claude -p --dangerously-skip-permissions", + timeout=0, + on_stdout=print, + on_stderr=print, + ) + + print(f"\n Check your repository for branch: {branch_name}") + + await sbx.kill() + +if __name__ == "__main__": + asyncio.run(fix_code_issue()) +``` + +Run the script: + +```bash +python 05_fix_code_issue.py +``` + +{{< /tab >}} +{{< /tabs >}} + +> [!NOTE] +> +> This script may take a few minutes to run. + +Claude will analyze your repository and fix a code quality issue: + +```console {collapse=true} +Creating sandbox... + +Fixing a code quality issue... + +## Task Completed Successfully! + +I've successfully analyzed the repository and fixed a code quality issue. Here's what I did: + +### 1. Analysis +Found 1 SonarQube issues in the `` repository: +- **Unused variable** (MINOR severity, line 2) +- Useless assignment (MAJOR severity, line 2) +- For-of loop suggestion (MINOR severity, lines 5-7) + +### 2. Issue Fixed +**Location:** test.js:2 +**Issue:** Unused variable declaration +**SonarQube Rule:** javascript:S1481 +**What was wrong:** The line `const unusedVariable = "this is not used";` declared a variable that was never used anywhere in the function. + +### 3. Why This Fix is Safe +- The variable was only declared and assigned, never referenced +- Removing it has zero impact on functionality +- It eliminates dead code and improves code quality +- This is a textbook example of a code smell that should be removed + +### 4. Changes Made +- Created branch: `quality-fix-xxxxx` +- Removed the unused variable declaration +- Committed with a clear message explaining the fix and referencing the SonarQube rule + +**Commit SHA:** `xxxxxxxxxxxxxxxxxxxxxxxxx` +**Branch:** https://github.com///tree/quality-fix-xxxxx + +The fix is ready for review and can be merged to improve the code quality of the repository! +``` + +You can now use GitHub and SonarQube MCP tools in the same workflow to read +files, make code changes, and commit them. + +### Step 6: Create quality-gated pull requests + +Finally, let's build the complete workflow: analyze quality, fix issues, +and create a PR only if improvements are made. + +{{< tabs group="language" >}} +{{< tab name="TypeScript" >}} + +Create `06-quality-gated-pr.ts`: + +```typescript +import "dotenv/config"; +import { Sandbox } from "e2b"; + +async function qualityGatedPR() { + console.log("Creating sandbox for quality-gated PR workflow...\n"); + + const sbx = await Sandbox.betaCreate({ + envs: { + ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY!, + GITHUB_TOKEN: process.env.GITHUB_TOKEN!, + SONARQUBE_TOKEN: process.env.SONARQUBE_TOKEN!, + }, + mcp: { + githubOfficial: { + githubPersonalAccessToken: process.env.GITHUB_TOKEN!, + }, + sonarqube: { + org: process.env.SONARQUBE_ORG!, + token: process.env.SONARQUBE_TOKEN!, + url: "https://sonarcloud.io", + }, + }, + }); + + const mcpUrl = sbx.betaGetMcpUrl(); + const mcpToken = await sbx.betaGetMcpToken(); + + await new Promise((resolve) => setTimeout(resolve, 1000)); + + await sbx.commands.run( + `claude mcp add --transport http e2b-mcp-gateway ${mcpUrl} --header "Authorization: Bearer ${mcpToken}"`, + { timeoutMs: 0, onStdout: console.log, onStderr: console.log }, + ); + + const repoPath = `${process.env.GITHUB_OWNER}/${process.env.GITHUB_REPO}`; + const branchName = `quality-improvements-${Date.now()}`; + + console.log("\nRunning quality-gated PR workflow...\n"); + + const prompt = `You are a code quality engineer. Using GitHub and SonarQube MCP tools: + + STEP 1: ANALYSIS + - Get current code quality status from SonarQube for "${repoPath}" + - Record the current number of bugs, code smells, and vulnerabilities + - Identify 1-3 issues that you can confidently fix + + STEP 2: FIX ISSUES + - Create branch "${branchName}" + - For each issue you're fixing: + * Read the file with the issue + * Make the fix + * Commit with a descriptive message + - Only fix issues where you're 100% confident the fix is correct + + STEP 3: VERIFICATION + - After your fixes, check if quality metrics would improve + - Calculate: Would this reduce bugs/smells/vulnerabilities? + + STEP 4: QUALITY GATE + - Only proceed if your changes improve quality + - If quality would not improve, explain why and stop + + STEP 5: CREATE PR (only if quality gate passes) + - Create a pull request from "${branchName}" to main + - Title: "Quality improvements: [describe what you fixed]" + - Description should include: + * What issues you fixed + * Before/after quality metrics + * Why these fixes improve code quality + - Add a comment with detailed SonarQube analysis + + Be thorough and explain your decisions at each step.`; + + await sbx.commands.run( + `echo '${prompt.replace(/'/g, "'\\''")}' | claude -p --dangerously-skip-permissions`, + { + timeoutMs: 0, + onStdout: console.log, + onStderr: console.log, + }, + ); + + console.log(`\n Workflow complete! Check ${repoPath} for new pull request.`); + + await sbx.kill(); +} + +qualityGatedPR().catch(console.error); +``` + +Run the script: + +```bash +npx tsx 06-quality-gated-pr.ts +``` + +{{< /tab >}} +{{< tab name="Python" >}} + +Create `06_quality_gated_pr.py`: + +```python +import os +import asyncio +import time +from dotenv import load_dotenv +from e2b import AsyncSandbox + +load_dotenv() + +async def quality_gated_pr(): + print("Creating sandbox for quality-gated PR workflow...\n") + + sbx = await AsyncSandbox.beta_create( + envs={ + "ANTHROPIC_API_KEY": os.getenv("ANTHROPIC_API_KEY"), + "GITHUB_TOKEN": os.getenv("GITHUB_TOKEN"), + "SONARQUBE_TOKEN": os.getenv("SONARQUBE_TOKEN"), + }, + mcp={ + "githubOfficial": { + "githubPersonalAccessToken": os.getenv("GITHUB_TOKEN"), + }, + "sonarqube": { + "org": os.getenv("SONARQUBE_ORG"), + "token": os.getenv("SONARQUBE_TOKEN"), + "url": "https://sonarcloud.io", + }, + }, + ) + + mcp_url = sbx.beta_get_mcp_url() + mcp_token = await sbx.beta_get_mcp_token() + + await asyncio.sleep(1) + + await sbx.commands.run( + f'claude mcp add --transport http e2b-mcp-gateway {mcp_url} --header "Authorization: Bearer {mcp_token}"', + timeout=0, + on_stdout=print, + on_stderr=print, + ) + + repo_path = f"{os.getenv('GITHUB_OWNER')}/{os.getenv('GITHUB_REPO')}" + branch_name = f"quality-improvements-{int(time.time() * 1000)}" + + print("\nRunning quality-gated PR workflow...\n") + + prompt = f"""You are a code quality engineer. Using GitHub and SonarQube MCP tools: + + STEP 1: ANALYSIS + - Get current code quality status from SonarQube for "{repo_path}" + - Record the current number of bugs, code smells, and vulnerabilities + - Identify 1-3 issues that you can confidently fix + + STEP 2: FIX ISSUES + - Create branch "{branch_name}" + - For each issue you are fixing: + Read the file with the issue + Make the fix + Commit with a descriptive message + - Only fix issues where you are 100 percent confident the fix is correct + + STEP 3: VERIFICATION + - After your fixes, check if quality metrics would improve + - Calculate: Would this reduce bugs/smells/vulnerabilities? + + STEP 4: QUALITY GATE + - Only proceed if your changes improve quality + - If quality would not improve, explain why and stop + + STEP 5: CREATE PR (only if quality gate passes) + - Create a pull request from "{branch_name}" to main + - Title: "Quality improvements: [describe what you fixed]" + - Description should include: + What issues you fixed + Before/after quality metrics + Why these fixes improve code quality + - Add a comment with detailed SonarQube analysis + + Be thorough and explain your decisions at each step.""" + + await sbx.commands.run( + f"echo '{prompt.replace(chr(39), chr(39) + chr(92) + chr(39) + chr(39))}' | claude -p --dangerously-skip-permissions", + timeout=0, + on_stdout=print, + on_stderr=print, + ) + + print(f"\n Workflow complete! Check {repo_path} for new pull request.") + + await sbx.kill() + +if __name__ == "__main__": + asyncio.run(quality_gated_pr()) +``` + +Run the script: + +```bash +python 06_quality_gated_pr.py +``` + +{{< /tab >}} +{{< /tabs >}} + +> [!NOTE] +> +> This script may take a few minutes to run. + +Claude will run the entire workflow, creating a quality improvement +and opening a PR in GitHub: + +```console {collapse=true} +Creating sandbox for quality-gated PR workflow... + +Running quality-gated PR workflow... + +## Complete - Code Quality Improvement Successfully Executed + +### Summary of Work Completed + +**Pull Request Created:** https://github.com///pull/1 + +--- + +### STEP 1: ANALYSIS + +**Current State:** +- **Bugs:** 0 +- **Vulnerabilities:** 0 +- **Code Smells:** 1 +- **Lines of Code:** 12 +- **All Ratings:** A (Reliability, Security, Maintainability) + +**Issues Identified (all in test.js):** +1. xxxxxxxxx - Unused variable declaration (MINOR) + +--- + +### STEP 2: FIX ISSUES + +**Branch Created:** `quality-improvements-xxxxxxxx` + +**Changes Made:** +1. **Removed unused variable** (line 2) - Eliminated dead code that served no purpose +2. **Modernized loop pattern** (lines 5-7) - Converted `for (let i = 0; i < items.length; i++)` to `for (const item of items)` + +**Commit:** xxxxxxxxxx + +--- + +### STEP 3: VERIFICATION + +**Expected Impact:** +- Code Smells: 1 → 0 (100% reduction) +- Bugs: 0 → 0 (maintained) +- Vulnerabilities: 0 → 0 (maintained) +- All quality ratings maintained at A + +--- + +### STEP 4: QUALITY GATE PASSED + +**Decision Criteria Met:** +- ✅ Reduces code smells by 100% +- ✅ No new bugs or vulnerabilities introduced +- ✅ Code is more readable and maintainable +- ✅ Follows modern JavaScript best practices +- ✅ All fixes are low-risk refactorings with no behavioral changes + +--- + +### STEP 5: CREATE PR + +**Pull Request Details:** +- **Number:** #1 +- **Title:** Quality improvements: Remove unused variable and modernize for loop +- **Branch:** quality-improvements-xxxxxxxx → main +- **URL:** https://github.com//pull/1 + +**PR Includes:** +- Comprehensive description with before/after metrics +- Detailed SonarQube analysis comment with issue breakdown +- Code comparison showing improvements +- Quality metrics table + +The pull request is now ready for review and merge! +``` + +You've now built a complete, multi-step workflow with conditional logic. +Claude analyzes quality with SonarQube, makes fixes using GitHub tools, +verifies improvements, and only creates a PR if quality actually improves. + +### Step 7: Add error handling + +Production workflows need error handling. Let's make the workflow more robust. + +{{< tabs group="language" >}} +{{< tab name="TypeScript" >}} + +Create `07-robust-workflow.ts`: + +```typescript +import "dotenv/config"; +import { Sandbox } from "e2b"; + +async function robustWorkflow() { + let sbx: Sandbox | undefined; + + try { + console.log("Creating sandbox...\n"); + + sbx = await Sandbox.betaCreate({ + envs: { + ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY!, + GITHUB_TOKEN: process.env.GITHUB_TOKEN!, + SONARQUBE_TOKEN: process.env.SONARQUBE_TOKEN!, + }, + mcp: { + githubOfficial: { + githubPersonalAccessToken: process.env.GITHUB_TOKEN!, + }, + sonarqube: { + org: process.env.SONARQUBE_ORG!, + token: process.env.SONARQUBE_TOKEN!, + url: "https://sonarcloud.io", + }, + }, + }); + + const mcpUrl = sbx.betaGetMcpUrl(); + const mcpToken = await sbx.betaGetMcpToken(); + + await new Promise((resolve) => setTimeout(resolve, 1000)); + + await sbx.commands.run( + `claude mcp add --transport http e2b-mcp-gateway ${mcpUrl} --header "Authorization: Bearer ${mcpToken}"`, + { timeoutMs: 0, onStdout: console.log, onStderr: console.log }, + ); + + const repoPath = `${process.env.GITHUB_OWNER}/${process.env.GITHUB_REPO}`; + + console.log("\nRunning workflow with error handling...\n"); + + const prompt = `Run a quality improvement workflow for "${repoPath}". + + ERROR HANDLING RULES: + 1. If SonarQube is unreachable, explain the error and stop gracefully + 2. If GitHub API fails, retry once, then explain and stop + 3. If no fixable issues are found, explain why and exit (this is not an error) + 4. If file modifications fail, explain which file and why + 5. At each step, check for errors before proceeding + + Run the workflow and handle any errors you encounter professionally.`; + + await sbx.commands.run( + `echo '${prompt.replace(/'/g, "'\\''")}' | claude -p --dangerously-skip-permissions`, + { + timeoutMs: 0, + onStdout: console.log, + onStderr: console.log, + }, + ); + + console.log("\n Workflow completed"); + } catch (error) { + const err = error as Error; + console.error("\n Workflow failed:", err.message); + + if (err.message.includes("403")) { + console.error("\n Check your E2B account has MCP gateway access"); + } else if (err.message.includes("401")) { + console.error("\n Check your API tokens are valid"); + } else if (err.message.includes("Credit balance")) { + console.error("\n Check your Anthropic API credit balance"); + } + + process.exit(1); + } finally { + if (sbx) { + console.log("\n Cleaning up sandbox..."); + await sbx.kill(); + } + } +} + +robustWorkflow().catch(console.error); +``` + +Run the script: + +```bash +npx tsx 07-robust-workflow.ts +``` + +{{< /tab >}} +{{< tab name="Python" >}} + +Create `07_robust_workflow.py`: + +```python +import os +import asyncio +import sys +from dotenv import load_dotenv +from e2b import AsyncSandbox + +load_dotenv() + +async def robust_workflow(): + sbx = None + + try: + print("Creating sandbox...\n") + + sbx = await AsyncSandbox.beta_create( + envs={ + "ANTHROPIC_API_KEY": os.getenv("ANTHROPIC_API_KEY"), + "GITHUB_TOKEN": os.getenv("GITHUB_TOKEN"), + "SONARQUBE_TOKEN": os.getenv("SONARQUBE_TOKEN"), + }, + mcp={ + "githubOfficial": { + "githubPersonalAccessToken": os.getenv("GITHUB_TOKEN"), + }, + "sonarqube": { + "org": os.getenv("SONARQUBE_ORG"), + "token": os.getenv("SONARQUBE_TOKEN"), + "url": "https://sonarcloud.io", + }, + }, + ) + + mcp_url = sbx.beta_get_mcp_url() + mcp_token = await sbx.beta_get_mcp_token() + + await asyncio.sleep(1) + + await sbx.commands.run( + f'claude mcp add --transport http e2b-mcp-gateway {mcp_url} --header "Authorization: Bearer {mcp_token}"', + timeout=0, # Fixed: was timeout_ms + on_stdout=print, + on_stderr=print, + ) + + repo_path = f"{os.getenv('GITHUB_OWNER')}/{os.getenv('GITHUB_REPO')}" + + print("\nRunning workflow with error handling...\n") + + prompt = f"""Run a quality improvement workflow for "{repo_path}". + + ERROR HANDLING RULES: + 1. If SonarQube is unreachable, explain the error and stop gracefully + 2. If GitHub API fails, retry once, then explain and stop + 3. If no fixable issues are found, explain why and exit (this is not an error) + 4. If file modifications fail, explain which file and why + 5. At each step, check for errors before proceeding + + Run the workflow and handle any errors you encounter professionally.""" + + await sbx.commands.run( + f"echo '{prompt}' | claude -p --dangerously-skip-permissions", + timeout=0, + on_stdout=print, + on_stderr=print, + ) + + print("\n Workflow completed") + + except Exception as error: + print(f"\n✗ Workflow failed: {str(error)}") + + error_msg = str(error) + if "403" in error_msg: + print("\n Check your E2B account has MCP gateway access") + elif "401" in error_msg: + print("\n Check your API tokens are valid") + elif "Credit balance" in error_msg: + print("\n Check your Anthropic API credit balance") + + sys.exit(1) + + finally: + if sbx: + print("\n Cleaning up sandbox...") + await sbx.kill() + +if __name__ == "__main__": + asyncio.run(robust_workflow()) +``` + +Run the script: + +```bash +python 07_robust_workflow.py +``` + +{{< /tab >}} +{{< /tabs >}} + +Claude will run the entire workflow, and if it encounters an error, respond +with robust error messaging. + +### Next steps + +In the next section, you'll customize your workflow for your needs. + +## Customize a code quality check workflow + +Now that you understand the basics of automating code quality workflows with +GitHub and SonarQube in E2B sandboxes, you can customize the workflow +for your needs. + +### Focus on specific quality issues + +Modify the prompt to prioritize certain issue types: + +{{< tabs group="language" >}} +{{< tab name="TypeScript" >}} + +```typescript +const prompt = `Using SonarQube and GitHub MCP tools: + +Focus only on: +- Security vulnerabilities (CRITICAL priority) +- Bugs (HIGH priority) +- Skip code smells for this iteration + +Analyze "${repoPath}" and fix the highest priority issues first.`; +``` + +{{< /tab >}} +{{< tab name="Python" >}} + +```python +prompt = f"""Using SonarQube and GitHub MCP tools: + +Focus only on: +- Security vulnerabilities (CRITICAL priority) +- Bugs (HIGH priority) +- Skip code smells for this iteration + +Analyze "{repo_path}" and fix the highest priority issues first.""" +``` + +{{< /tab >}} +{{< /tabs >}} + +### Integrate with CI/CD + +Add this workflow to GitHub Actions to run automatically on pull requests: + +{{< tabs group="language" >}} +{{< tab name="TypeScript" >}} + +```yaml +name: Automated quality checks +on: + pull_request: + types: [opened, synchronize] + +jobs: + quality: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@{{% param "checkout_action_version" %}} + - uses: actions/setup-node@v5 + with: + node-version: "24" + - run: npm install + - run: npx tsx 06-quality-gated-pr.ts + env: + E2B_API_KEY: ${{ secrets.E2B_API_KEY }} + ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + SONARQUBE_TOKEN: ${{ secrets.SONARQUBE_TOKEN }} + GITHUB_OWNER: ${{ github.repository_owner }} + GITHUB_REPO: ${{ github.event.repository.name }} + SONARQUBE_ORG: your-org-key +``` + +{{< /tab >}} +{{< tab name="Python" >}} + +```yaml +name: Automated quality checks +on: + pull_request: + types: [opened, synchronize] + +jobs: + quality: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@{{% param "checkout_action_version" %}} + - uses: actions/setup-python@v6 + with: + python-version: "3.14" + - run: pip install e2b python-dotenv + - run: python 06_quality_gated_pr.py + env: + E2B_API_KEY: ${{ secrets.E2B_API_KEY }} + ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + SONARQUBE_TOKEN: ${{ secrets.SONARQUBE_TOKEN }} + GITHUB_OWNER: ${{ github.repository_owner }} + GITHUB_REPO: ${{ github.event.repository.name }} + SONARQUBE_ORG: your-org-key +``` + +{{< /tab >}} +{{< /tabs >}} + +### Filter by file patterns + +Target specific parts of your codebase: + +{{< tabs group="language" >}} +{{< tab name="TypeScript" >}} + +```typescript +const prompt = `Analyze code quality but only consider: +- Files in src/**/*.js +- Exclude test files (*.test.js, *.spec.js) +- Exclude build artifacts in dist/ + +Focus on production code only.`; +``` + +{{< /tab >}} +{{< tab name="Python" >}} + +```python +prompt = """Analyze code quality but only consider: +- Files in src/**/*.js +- Exclude test files (*.test.js, *.spec.js) +- Exclude build artifacts in dist/ + +Focus on production code only.""" +``` + +{{< /tab >}} +{{< /tabs >}} + +### Set quality thresholds + +Define when PRs should be created: + +{{< tabs group="language" >}} +{{< tab name="TypeScript" >}} + +```typescript +const prompt = `Quality gate thresholds: +- Only create PR if: + * Bug count decreases by at least 1 + * No new security vulnerabilities introduced + * Code coverage does not decrease + * Technical debt reduces by at least 15 minutes + +If changes do not meet these thresholds, explain why and skip PR creation.`; +``` + +{{< /tab >}} +{{< tab name="Python" >}} + +```python +prompt = """Quality gate thresholds: +- Only create PR if: + * Bug count decreases by at least 1 + * No new security vulnerabilities introduced + * Code coverage does not decrease + * Technical debt reduces by at least 15 minutes + +If changes do not meet these thresholds, explain why and skip PR creation.""" +``` + +{{< /tab >}} +{{< /tabs >}} + +### Next steps + +Learn how to troubleshoot common issues. + +## Troubleshoot code quality workflows + +This page covers common issues you might encounter when building code quality +workflows with E2B sandboxes and MCP servers, along with their solutions. + +If you're experiencing problems not covered here, check the +[E2B documentation](https://e2b.dev/docs). + +### MCP tools not available + +Issue: Claude reports `I don't have any MCP tools available`. + +Solution: + +1. Verify you're using the authorization header: + + ```plaintext + --header "Authorization: Bearer ${mcpToken}" + ``` + +2. Check you're waiting for MCP initialization. + + ```typescript + // typescript + await new Promise((resolve) => setTimeout(resolve, 1000)); + ``` + + ```python + # python + await asyncio.sleep(1) + ``` + +3. Ensure credentials are in both `envs` and `mcp` configuration: + + ```typescript + // typescript + const sbx = await Sandbox.betaCreate({ + envs: { + ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY!, + GITHUB_TOKEN: process.env.GITHUB_TOKEN!, + SONARQUBE_TOKEN: process.env.SONARQUBE_TOKEN!, + }, + mcp: { + githubOfficial: { + githubPersonalAccessToken: process.env.GITHUB_TOKEN!, + }, + sonarqube: { + org: process.env.SONARQUBE_ORG!, + token: process.env.SONARQUBE_TOKEN!, + url: "https://sonarcloud.io", + }, + }, + }); + ``` + + ```python + # python + sbx = await AsyncSandbox.beta_create( + envs={ + "ANTHROPIC_API_KEY": os.getenv("ANTHROPIC_API_KEY"), + "GITHUB_TOKEN": os.getenv("GITHUB_TOKEN"), + "SONARQUBE_TOKEN": os.getenv("SONARQUBE_TOKEN"), + }, + mcp={ + "githubOfficial": { + "githubPersonalAccessToken": os.getenv("GITHUB_TOKEN"), + }, + "sonarqube": { + "org": os.getenv("SONARQUBE_ORG"), + "token": os.getenv("SONARQUBE_TOKEN"), + "url": "https://sonarcloud.io", + }, + }, + ) + ``` + +4. Verify your API tokens are valid and have proper scopes. + +### GitHub tools work but SonarQube doesn't + +Issue: GitHub MCP tools load but SonarQube tools don't appear. + +Solution: SonarQube MCP server requires GitHub to be configured simultaneously. +Always include both servers in your sandbox configuration, even if you're only +testing one. + +{{< tabs group="language" >}} +{{< tab name="TypeScript" >}} + +```typescript +// Include both servers even if only using one +const sbx = await Sandbox.betaCreate({ + envs: { + ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY!, + GITHUB_TOKEN: process.env.GITHUB_TOKEN!, + SONARQUBE_TOKEN: process.env.SONARQUBE_TOKEN!, + }, + mcp: { + githubOfficial: { + githubPersonalAccessToken: process.env.GITHUB_TOKEN!, + }, + sonarqube: { + org: process.env.SONARQUBE_ORG!, + token: process.env.SONARQUBE_TOKEN!, + url: "https://sonarcloud.io", + }, + }, +}); +``` + +{{< /tab >}} +{{< tab name="Python" >}} + +```python +# Include both servers even if only using one +sbx = await AsyncSandbox.beta_create( + envs={ + "ANTHROPIC_API_KEY": os.getenv("ANTHROPIC_API_KEY"), + "GITHUB_TOKEN": os.getenv("GITHUB_TOKEN"), + "SONARQUBE_TOKEN": os.getenv("SONARQUBE_TOKEN"), + }, + mcp={ + "githubOfficial": { + "githubPersonalAccessToken": os.getenv("GITHUB_TOKEN"), + }, + "sonarqube": { + "org": os.getenv("SONARQUBE_ORG"), + "token": os.getenv("SONARQUBE_TOKEN"), + "url": "https://sonarcloud.io", + }, + }, +) +``` + +{{< /tab >}} +{{< /tabs >}} + +### Claude can't access private repositories + +Issue: "I don't have access to that repository". + +Solution: + +1. Verify your GitHub token has `repo` scope (not just `public_repo`). +2. Test with a public repository first. +3. Ensure the repository owner and name are correct in your `.env`: + + {{< tabs group="language" >}} + {{< tab name="TypeScript" >}} + + ```plaintext + GITHUB_OWNER=your_github_username + GITHUB_REPO=your_repository_name + ``` + + {{< /tab >}} + {{< tab name="Python" >}} + + ```plaintext + GITHUB_OWNER=your_github_username + GITHUB_REPO=your_repository_name + ``` + + {{< /tab >}} + {{< /tabs >}} + +### Workflow times out or runs too long + +Issue: Workflow doesn't complete or Claude credits run out. + +Solutions: + +1. Use `timeoutMs: 0` (TypeScript) or `timeout_ms=0` (Python) for complex workflows to allow unlimited time: + + {{< tabs group="language" >}} + {{< tab name="TypeScript" >}} + + ```typescript + await sbx.commands.run( + `echo '${prompt}' | claude -p --dangerously-skip-permissions`, + { + timeoutMs: 0, // No timeout + onStdout: console.log, + onStderr: console.log, + }, + ); + ``` + + {{< /tab >}} + {{< tab name="Python" >}} + + ```python + await sbx.commands.run( + f"echo '{prompt}' | claude -p --dangerously-skip-permissions", + timeout_ms=0, # No timeout + on_stdout=print, + on_stderr=print, + ) + ``` + + {{< /tab >}} + {{< /tabs >}} + +2. Break complex workflows into smaller, focused tasks. +3. Monitor your Anthropic API credit usage. +4. Add checkpoints in prompts: "After each step, show progress before continuing". + +### Sandbox cleanup errors + +Issue: Sandboxes aren't being cleaned up properly, leading to resource exhaustion. + +Solution: Always use proper error handling with cleanup in the `finally` block: + +{{< tabs group="language" >}} +{{< tab name="TypeScript" >}} + +```typescript +async function robustWorkflow() { + let sbx: Sandbox | undefined; + + try { + sbx = await Sandbox.betaCreate({ + // ... configuration + }); + + // ... workflow logic + } catch (error) { + console.error("Workflow failed:", error); + process.exit(1); + } finally { + if (sbx) { + console.log("Cleaning up sandbox..."); + await sbx.kill(); + } + } +} +``` + +{{< /tab >}} +{{< tab name="Python" >}} + +```python +async def robust_workflow(): + sbx = None + + try: + sbx = await AsyncSandbox.beta_create( + # ... configuration + ) + + # ... workflow logic + + except Exception as error: + print(f"Workflow failed: {error}") + sys.exit(1) + finally: + if sbx: + print("Cleaning up sandbox...") + await sbx.kill() +``` + +{{< /tab >}} +{{< /tabs >}} + +### Environment variable not loading + +Issue: Script fails with "undefined" or "None" for environment variables. + +Solution: + +{{< tabs group="language" >}} +{{< tab name="TypeScript" >}} + +1. Ensure `dotenv` is loaded at the top of your file: + + ```typescript + import "dotenv/config"; + ``` + +2. Verify the `.env` file is in the same directory as your script. + +3. Check variable names match exactly (case-sensitive): + + ```typescript + // .env file + GITHUB_TOKEN = ghp_xxxxx; + + // In code + process.env.GITHUB_TOKEN; // Correct + process.env.github_token; // Wrong - case doesn't match + ``` + + {{< /tab >}} + {{< tab name="Python" >}} + 1. Ensure `dotenv` is loaded at the top of your file: + + ```python + from dotenv import load_dotenv + load_dotenv() + ``` + + 2. Verify the `.env` file is in the same directory as your script. + + 3. Check variable names match exactly (case-sensitive): + + ```python + # .env file + GITHUB_TOKEN=ghp_xxxxx + + # In code + os.getenv("GITHUB_TOKEN") # Correct + os.getenv("github_token") # Wrong - case doesn't match + ``` + + {{< /tab >}} + {{< /tabs >}} + +### SonarQube returns empty results + +Issue: SonarQube analysis returns no projects or issues. + +Solution: + +1. Verify your SonarCloud organization key is correct. +2. Ensure you have at least one project configured in SonarCloud. +3. Check that your SonarQube token has the necessary permissions. +4. Confirm your project has been analyzed at least once in SonarCloud. diff --git a/content/guides/github-sonarqube-sandbox/customize.md b/content/guides/github-sonarqube-sandbox/customize.md deleted file mode 100644 index 832ad51a2e14..000000000000 --- a/content/guides/github-sonarqube-sandbox/customize.md +++ /dev/null @@ -1,182 +0,0 @@ ---- -title: Customize a code quality check workflow -linkTitle: Customize workflow -summary: Adapt your GitHub and SonarQube workflow to focus on specific quality issues, integrate with CI/CD, and set custom thresholds. -description: Learn how to customize prompts for specific quality issues, filter by file patterns, set quality thresholds, and integrate your workflow with GitHub Actions for automated code quality checks. -keywords: sonarqube, e2b, mcp, github actions, code quality, customization, ci/cd -weight: 20 ---- - -Now that you understand the basics of automating code quality workflows with -GitHub and SonarQube in E2B sandboxes, you can customize the workflow -for your needs. - -## Focus on specific quality issues - -Modify the prompt to prioritize certain issue types: - -{{< tabs group="language" >}} -{{< tab name="TypeScript" >}} - -```typescript -const prompt = `Using SonarQube and GitHub MCP tools: - -Focus only on: -- Security vulnerabilities (CRITICAL priority) -- Bugs (HIGH priority) -- Skip code smells for this iteration - -Analyze "${repoPath}" and fix the highest priority issues first.`; -``` - -{{< /tab >}} -{{< tab name="Python" >}} - -```python -prompt = f"""Using SonarQube and GitHub MCP tools: - -Focus only on: -- Security vulnerabilities (CRITICAL priority) -- Bugs (HIGH priority) -- Skip code smells for this iteration - -Analyze "{repo_path}" and fix the highest priority issues first.""" -``` - -{{< /tab >}} -{{< /tabs >}} - -## Integrate with CI/CD - -Add this workflow to GitHub Actions to run automatically on pull requests: - -{{< tabs group="language" >}} -{{< tab name="TypeScript" >}} - -```yaml -name: Automated quality checks -on: - pull_request: - types: [opened, synchronize] - -jobs: - quality: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@{{% param "checkout_action_version" %}} - - uses: actions/setup-node@v5 - with: - node-version: "24" - - run: npm install - - run: npx tsx 06-quality-gated-pr.ts - env: - E2B_API_KEY: ${{ secrets.E2B_API_KEY }} - ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - SONARQUBE_TOKEN: ${{ secrets.SONARQUBE_TOKEN }} - GITHUB_OWNER: ${{ github.repository_owner }} - GITHUB_REPO: ${{ github.event.repository.name }} - SONARQUBE_ORG: your-org-key -``` - -{{< /tab >}} -{{< tab name="Python" >}} - -```yaml -name: Automated quality checks -on: - pull_request: - types: [opened, synchronize] - -jobs: - quality: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@{{% param "checkout_action_version" %}} - - uses: actions/setup-python@v6 - with: - python-version: "3.14" - - run: pip install e2b python-dotenv - - run: python 06_quality_gated_pr.py - env: - E2B_API_KEY: ${{ secrets.E2B_API_KEY }} - ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - SONARQUBE_TOKEN: ${{ secrets.SONARQUBE_TOKEN }} - GITHUB_OWNER: ${{ github.repository_owner }} - GITHUB_REPO: ${{ github.event.repository.name }} - SONARQUBE_ORG: your-org-key -``` - -{{< /tab >}} -{{< /tabs >}} - -## Filter by file patterns - -Target specific parts of your codebase: - -{{< tabs group="language" >}} -{{< tab name="TypeScript" >}} - -```typescript -const prompt = `Analyze code quality but only consider: -- Files in src/**/*.js -- Exclude test files (*.test.js, *.spec.js) -- Exclude build artifacts in dist/ - -Focus on production code only.`; -``` - -{{< /tab >}} -{{< tab name="Python" >}} - -```python -prompt = """Analyze code quality but only consider: -- Files in src/**/*.js -- Exclude test files (*.test.js, *.spec.js) -- Exclude build artifacts in dist/ - -Focus on production code only.""" -``` - -{{< /tab >}} -{{< /tabs >}} - -## Set quality thresholds - -Define when PRs should be created: - -{{< tabs group="language" >}} -{{< tab name="TypeScript" >}} - -```typescript -const prompt = `Quality gate thresholds: -- Only create PR if: - * Bug count decreases by at least 1 - * No new security vulnerabilities introduced - * Code coverage does not decrease - * Technical debt reduces by at least 15 minutes - -If changes do not meet these thresholds, explain why and skip PR creation.`; -``` - -{{< /tab >}} -{{< tab name="Python" >}} - -```python -prompt = """Quality gate thresholds: -- Only create PR if: - * Bug count decreases by at least 1 - * No new security vulnerabilities introduced - * Code coverage does not decrease - * Technical debt reduces by at least 15 minutes - -If changes do not meet these thresholds, explain why and skip PR creation.""" -``` - -{{< /tab >}} -{{< /tabs >}} - -## Next steps - -Learn how to troubleshoot common issues. diff --git a/content/guides/github-sonarqube-sandbox/troubleshoot.md b/content/guides/github-sonarqube-sandbox/troubleshoot.md deleted file mode 100644 index 6ea50554acb0..000000000000 --- a/content/guides/github-sonarqube-sandbox/troubleshoot.md +++ /dev/null @@ -1,334 +0,0 @@ ---- -title: Troubleshoot code quality workflows -linkTitle: Troubleshoot -summary: Resolve common issues with E2B sandboxes, MCP server connections, and GitHub/SonarQube integration. -description: Solutions for MCP tools not loading, authentication errors, permission issues, workflow timeouts, and other common problems when building code quality workflows with E2B. -keywords: sonarqube, e2b, mcp, troubleshooting, debugging, authentication, code quality -weight: 30 ---- - -This page covers common issues you might encounter when building code quality -workflows with E2B sandboxes and MCP servers, along with their solutions. - -If you're experiencing problems not covered here, check the -[E2B documentation](https://e2b.dev/docs). - -## MCP tools not available - -Issue: Claude reports `I don't have any MCP tools available`. - -Solution: - -1. Verify you're using the authorization header: - - ```plaintext - --header "Authorization: Bearer ${mcpToken}" - ``` - -2. Check you're waiting for MCP initialization. - - ```typescript - // typescript - await new Promise((resolve) => setTimeout(resolve, 1000)); - ``` - - ```python - # python - await asyncio.sleep(1) - ``` - -3. Ensure credentials are in both `envs` and `mcp` configuration: - - ```typescript - // typescript - const sbx = await Sandbox.betaCreate({ - envs: { - ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY!, - GITHUB_TOKEN: process.env.GITHUB_TOKEN!, - SONARQUBE_TOKEN: process.env.SONARQUBE_TOKEN!, - }, - mcp: { - githubOfficial: { - githubPersonalAccessToken: process.env.GITHUB_TOKEN!, - }, - sonarqube: { - org: process.env.SONARQUBE_ORG!, - token: process.env.SONARQUBE_TOKEN!, - url: "https://sonarcloud.io", - }, - }, - }); - ``` - - ```python - # python - sbx = await AsyncSandbox.beta_create( - envs={ - "ANTHROPIC_API_KEY": os.getenv("ANTHROPIC_API_KEY"), - "GITHUB_TOKEN": os.getenv("GITHUB_TOKEN"), - "SONARQUBE_TOKEN": os.getenv("SONARQUBE_TOKEN"), - }, - mcp={ - "githubOfficial": { - "githubPersonalAccessToken": os.getenv("GITHUB_TOKEN"), - }, - "sonarqube": { - "org": os.getenv("SONARQUBE_ORG"), - "token": os.getenv("SONARQUBE_TOKEN"), - "url": "https://sonarcloud.io", - }, - }, - ) - ``` - -4. Verify your API tokens are valid and have proper scopes. - -## GitHub tools work but SonarQube doesn't - -Issue: GitHub MCP tools load but SonarQube tools don't appear. - -Solution: SonarQube MCP server requires GitHub to be configured simultaneously. -Always include both servers in your sandbox configuration, even if you're only -testing one. - -{{< tabs group="language" >}} -{{< tab name="TypeScript" >}} - -```typescript -// Include both servers even if only using one -const sbx = await Sandbox.betaCreate({ - envs: { - ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY!, - GITHUB_TOKEN: process.env.GITHUB_TOKEN!, - SONARQUBE_TOKEN: process.env.SONARQUBE_TOKEN!, - }, - mcp: { - githubOfficial: { - githubPersonalAccessToken: process.env.GITHUB_TOKEN!, - }, - sonarqube: { - org: process.env.SONARQUBE_ORG!, - token: process.env.SONARQUBE_TOKEN!, - url: "https://sonarcloud.io", - }, - }, -}); -``` - -{{< /tab >}} -{{< tab name="Python" >}} - -```python -# Include both servers even if only using one -sbx = await AsyncSandbox.beta_create( - envs={ - "ANTHROPIC_API_KEY": os.getenv("ANTHROPIC_API_KEY"), - "GITHUB_TOKEN": os.getenv("GITHUB_TOKEN"), - "SONARQUBE_TOKEN": os.getenv("SONARQUBE_TOKEN"), - }, - mcp={ - "githubOfficial": { - "githubPersonalAccessToken": os.getenv("GITHUB_TOKEN"), - }, - "sonarqube": { - "org": os.getenv("SONARQUBE_ORG"), - "token": os.getenv("SONARQUBE_TOKEN"), - "url": "https://sonarcloud.io", - }, - }, -) -``` - -{{< /tab >}} -{{< /tabs >}} - -## Claude can't access private repositories - -Issue: "I don't have access to that repository". - -Solution: - -1. Verify your GitHub token has `repo` scope (not just `public_repo`). -2. Test with a public repository first. -3. Ensure the repository owner and name are correct in your `.env`: - - {{< tabs group="language" >}} - {{< tab name="TypeScript" >}} - - ```plaintext - GITHUB_OWNER=your_github_username - GITHUB_REPO=your_repository_name - ``` - - {{< /tab >}} - {{< tab name="Python" >}} - - ```plaintext - GITHUB_OWNER=your_github_username - GITHUB_REPO=your_repository_name - ``` - - {{< /tab >}} - {{< /tabs >}} - -## Workflow times out or runs too long - -Issue: Workflow doesn't complete or Claude credits run out. - -Solutions: - -1. Use `timeoutMs: 0` (TypeScript) or `timeout_ms=0` (Python) for complex workflows to allow unlimited time: - - {{< tabs group="language" >}} - {{< tab name="TypeScript" >}} - - ```typescript - await sbx.commands.run( - `echo '${prompt}' | claude -p --dangerously-skip-permissions`, - { - timeoutMs: 0, // No timeout - onStdout: console.log, - onStderr: console.log, - }, - ); - ``` - - {{< /tab >}} - {{< tab name="Python" >}} - - ```python - await sbx.commands.run( - f"echo '{prompt}' | claude -p --dangerously-skip-permissions", - timeout_ms=0, # No timeout - on_stdout=print, - on_stderr=print, - ) - ``` - - {{< /tab >}} - {{< /tabs >}} - -2. Break complex workflows into smaller, focused tasks. -3. Monitor your Anthropic API credit usage. -4. Add checkpoints in prompts: "After each step, show progress before continuing". - -## Sandbox cleanup errors - -Issue: Sandboxes aren't being cleaned up properly, leading to resource exhaustion. - -Solution: Always use proper error handling with cleanup in the `finally` block: - -{{< tabs group="language" >}} -{{< tab name="TypeScript" >}} - -```typescript -async function robustWorkflow() { - let sbx: Sandbox | undefined; - - try { - sbx = await Sandbox.betaCreate({ - // ... configuration - }); - - // ... workflow logic - } catch (error) { - console.error("Workflow failed:", error); - process.exit(1); - } finally { - if (sbx) { - console.log("Cleaning up sandbox..."); - await sbx.kill(); - } - } -} -``` - -{{< /tab >}} -{{< tab name="Python" >}} - -```python -async def robust_workflow(): - sbx = None - - try: - sbx = await AsyncSandbox.beta_create( - # ... configuration - ) - - # ... workflow logic - - except Exception as error: - print(f"Workflow failed: {error}") - sys.exit(1) - finally: - if sbx: - print("Cleaning up sandbox...") - await sbx.kill() -``` - -{{< /tab >}} -{{< /tabs >}} - -## Environment variable not loading - -Issue: Script fails with "undefined" or "None" for environment variables. - -Solution: - -{{< tabs group="language" >}} -{{< tab name="TypeScript" >}} - -1. Ensure `dotenv` is loaded at the top of your file: - - ```typescript - import "dotenv/config"; - ``` - -2. Verify the `.env` file is in the same directory as your script. - -3. Check variable names match exactly (case-sensitive): - - ```typescript - // .env file - GITHUB_TOKEN = ghp_xxxxx; - - // In code - process.env.GITHUB_TOKEN; // Correct - process.env.github_token; // Wrong - case doesn't match - ``` - - {{< /tab >}} - {{< tab name="Python" >}} - 1. Ensure `dotenv` is loaded at the top of your file: - - ```python - from dotenv import load_dotenv - load_dotenv() - ``` - - 2. Verify the `.env` file is in the same directory as your script. - - 3. Check variable names match exactly (case-sensitive): - - ```python - # .env file - GITHUB_TOKEN=ghp_xxxxx - - # In code - os.getenv("GITHUB_TOKEN") # Correct - os.getenv("github_token") # Wrong - case doesn't match - ``` - - {{< /tab >}} - {{< /tabs >}} - -## SonarQube returns empty results - -Issue: SonarQube analysis returns no projects or issues. - -Solution: - -1. Verify your SonarCloud organization key is correct. -2. Ensure you have at least one project configured in SonarCloud. -3. Check that your SonarQube token has the necessary permissions. -4. Confirm your project has been analyzed at least once in SonarCloud. diff --git a/content/guides/github-sonarqube-sandbox/workflow.md b/content/guides/github-sonarqube-sandbox/workflow.md deleted file mode 100644 index 7268e31eb1fc..000000000000 --- a/content/guides/github-sonarqube-sandbox/workflow.md +++ /dev/null @@ -1,1588 +0,0 @@ ---- -title: Build a code quality check workflow -linkTitle: Build workflow -summary: Learn to use GitHub and SonarQube MCP servers in E2B sandboxes through progressive examples. -description: Create E2B sandboxes, discover MCP tools, test individual operations, and build complete quality-gated PR workflows. -keywords: sonarqube, e2b, mcp, github, code quality, pull requests, workflow -weight: 10 ---- - -In this section, you'll build a complete code quality automation workflow -step-by-step. You'll start by creating an E2B sandbox with GitHub and -SonarQube MCP servers, then progressively add functionality until you have a -production-ready workflow that analyzes code quality and creates pull requests. - -By working through each step sequentially, you'll learn how MCP servers work, -how to interact with them through Claude, and how to chain operations together -to build powerful automation workflows. - -## Prerequisites - -Before you begin, make sure you have: - -- E2B account with [API access](https://e2b.dev/docs/api-key) -- [Anthropic API key](https://docs.claude.com/en/api/admin-api/apikeys/get-api-key) - - > [!NOTE] - > - > This example uses Claude CLI which comes pre-installed in E2B sandboxes, but you can adapt the example to work with other AI assistants of your choice. See [E2B's MCP documentation](https://e2b.dev/docs/mcp/quickstart) for alternative connection methods. - -- GitHub account with: - - A repository containing code to analyze - - [Personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) with `repo` scope -- SonarCloud account with: - - [Organization](https://docs.sonarsource.com/sonarqube-cloud/administering-sonarcloud/resources-structure/organization) created - - [Project configured](https://docs.sonarsource.com/sonarqube-community-build/project-administration/creating-and-importing-projects) for your repository - - [User token](https://docs.sonarsource.com/sonarqube-server/instance-administration/security/administering-tokens) generated -- Language runtime installed: - - TypeScript: [Node.js 18+](https://nodejs.org/en/download) - - Python: [Python 3.8+](https://www.python.org/downloads/) - -> [!NOTE] -> -> This guide uses Claude's `--dangerously-skip-permissions` flag to enable -> automated command execution in E2B sandboxes. This flag bypasses permission -> prompts, which is appropriate for isolated container environments like E2B -> where sandboxes are disposable and separate from your local machine. -> -> However, be aware that Claude can execute any commands within the sandbox, -> including accessing files and credentials available in that environment. Only -> use this approach with trusted code and workflows. For more information, -> see [Anthropic's guidance on container security](https://docs.anthropic.com/en/docs/claude-code/devcontainer). - -## Set up your project - -{{< tabs group="language" >}} -{{< tab name="TypeScript" >}} - -1. Create a new directory for your workflow and initialize Node.js: - - ```bash - mkdir github-sonarqube-workflow - cd github-sonarqube-workflow - npm init -y - ``` - -2. Open `package.json` and configure it for ES modules: - - ```json - { - "name": "github-sonarqube-workflow", - "version": "1.0.0", - "description": "Automated code quality workflow using E2B, GitHub, and SonarQube", - "type": "module", - "main": "quality-workflow.ts", - "scripts": { - "start": "tsx quality-workflow.ts" - }, - "keywords": ["e2b", "github", "sonarqube", "mcp", "code-quality"], - "author": "", - "license": "MIT" - } - ``` - -3. Install required dependencies: - - ```bash - npm install e2b dotenv - npm install -D typescript tsx @types/node - ``` - -4. Create a `.env` file in your project root: - - ```bash - touch .env - ``` - -5. Add your API keys and configuration, replacing the placeholders with your actual credentials: - - ```plaintext - E2B_API_KEY=your_e2b_api_key_here - ANTHROPIC_API_KEY=your_anthropic_api_key_here - GITHUB_TOKEN=ghp_your_personal_access_token_here - GITHUB_OWNER=your_github_username - GITHUB_REPO=your_repository_name - SONARQUBE_ORG=your_sonarcloud_org_key - SONARQUBE_TOKEN=your_sonarqube_user_token - SONARQUBE_URL=https://sonarcloud.io - ``` - -6. Protect your credentials by adding `.env` to `.gitignore`: - - ```bash - echo ".env" >> .gitignore - echo "node_modules/" >> .gitignore - ``` - -{{< /tab >}} -{{< tab name="Python" >}} - -1. Create a new directory for your workflow: - - ```bash - mkdir github-sonarqube-workflow - cd github-sonarqube-workflow - ``` - -2. Create a virtual environment and activate it: - - ```bash - python3 -m venv venv - source venv/bin/activate # On Windows: venv\Scripts\activate - ``` - -3. Install required dependencies: - - ```bash - pip install e2b python-dotenv - ``` - -4. Create a `.env` file in your project root: - - ```bash - touch .env - ``` - -5. Add your API keys and configuration, replacing the placeholders with your actual credentials: - - ```plaintext - E2B_API_KEY=your_e2b_api_key_here - ANTHROPIC_API_KEY=your_anthropic_api_key_here - GITHUB_TOKEN=ghp_your_personal_access_token_here - GITHUB_OWNER=your_github_username - GITHUB_REPO=your_repository_name - SONARQUBE_ORG=your_sonarcloud_org_key - SONARQUBE_TOKEN=your_sonarqube_user_token - SONARQUBE_URL=https://sonarcloud.io - ``` - -6. Protect your credentials by adding `.env` to `.gitignore`: - - ```bash - echo ".env" >> .gitignore - echo "venv/" >> .gitignore - echo "__pycache__/" >> .gitignore - ``` - -{{< /tab >}} -{{< /tabs >}} - -## Step 1: Create your first sandbox - -Let's start by creating a sandbox and verifying the MCP servers are configured correctly. - -{{< tabs group="language" >}} -{{< tab name="TypeScript" >}} - -Create a file named `01-test-connection.ts` in your project root: - -```typescript -import "dotenv/config"; -import { Sandbox } from "e2b"; - -async function testConnection() { - console.log( - "Creating E2B sandbox with GitHub and SonarQube MCP servers...\n", - ); - - const sbx = await Sandbox.betaCreate({ - envs: { - ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY!, - GITHUB_TOKEN: process.env.GITHUB_TOKEN!, - SONARQUBE_TOKEN: process.env.SONARQUBE_TOKEN!, - }, - mcp: { - githubOfficial: { - githubPersonalAccessToken: process.env.GITHUB_TOKEN!, - }, - sonarqube: { - org: process.env.SONARQUBE_ORG!, - token: process.env.SONARQUBE_TOKEN!, - url: "https://sonarcloud.io", - }, - }, - }); - - const mcpUrl = sbx.betaGetMcpUrl(); - const mcpToken = await sbx.betaGetMcpToken(); - - console.log(" Sandbox created successfully!"); - console.log(`MCP Gateway URL: ${mcpUrl}\n`); - - // Wait for MCP initialization - await new Promise((resolve) => setTimeout(resolve, 1000)); - - // Configure Claude to use the MCP gateway - console.log("Connecting Claude CLI to MCP gateway..."); - await sbx.commands.run( - `claude mcp add --transport http e2b-mcp-gateway ${mcpUrl} --header "Authorization: Bearer ${mcpToken}"`, - { - timeoutMs: 0, - onStdout: console.log, - onStderr: console.log, - }, - ); - - console.log("\nœConnection successful! Cleaning up..."); - await sbx.kill(); -} - -testConnection().catch(console.error); -``` - -Run this script to verify your setup: - -```bash -npx tsx 01-test-connection.ts -``` - -{{< /tab >}} -{{< tab name="Python" >}} - -Create a file named `01_test_connection.py` in your project root: - -```python -import os -import asyncio -from dotenv import load_dotenv -from e2b import AsyncSandbox - -load_dotenv() - -async def test_connection(): - print("Creating E2B sandbox with GitHub and SonarQube MCP servers...\n") - - sbx = await AsyncSandbox.beta_create( - envs={ - "ANTHROPIC_API_KEY": os.getenv("ANTHROPIC_API_KEY"), - "GITHUB_TOKEN": os.getenv("GITHUB_TOKEN"), - "SONARQUBE_TOKEN": os.getenv("SONARQUBE_TOKEN"), - }, - mcp={ - "githubOfficial": { - "githubPersonalAccessToken": os.getenv("GITHUB_TOKEN"), - }, - "sonarqube": { - "org": os.getenv("SONARQUBE_ORG"), - "token": os.getenv("SONARQUBE_TOKEN"), - "url": "https://sonarcloud.io", - }, - }, - ) - - mcp_url = sbx.beta_get_mcp_url() - mcp_token = await sbx.beta_get_mcp_token() - - print(" Sandbox created successfully!") - print(f"MCP Gateway URL: {mcp_url}\n") - - # Wait for MCP initialization - await asyncio.sleep(1) - - # Configure Claude to use the MCP gateway - print("Connecting Claude CLI to MCP gateway...") - await sbx.commands.run( - f'claude mcp add --transport http e2b-mcp-gateway {mcp_url} --header "Authorization: Bearer {mcp_token}"', - timeout=0, - on_stdout=print, - on_stderr=print, - ) - - print("\n Connection successful! Cleaning up...") - await sbx.kill() - -if __name__ == "__main__": - asyncio.run(test_connection()) -``` - -Run this script to verify your setup: - -```bash -python 01_test_connection.py -``` - -{{< /tab >}} -{{< /tabs >}} - -Your output should look similar to the following example: - -```console {collapse=true} -Creating E2B sandbox with GitHub and SonarQube MCP servers... - -✓ Sandbox created successfully! -MCP Gateway URL: https://50005-xxxxx.e2b.app/mcp - -Connecting Claude CLI to MCP gateway... -Added HTTP MCP server e2b-mcp-gateway with URL: https://50005-xxxxx.e2b.app/mcp to local config -Headers: { - "Authorization": "Bearer xxxxx-xxxx-xxxx" -} -File modified: /home/user/.claude.json [project: /home/user] - -✓ Connection successful! Cleaning up... -``` - -You've just learned how to create an E2B sandbox with multiple MCP servers -configured. The `betaCreate` method initializes a cloud environment -with Claude CLI and your specified MCP servers. - -## Step 2: Discover available MCP tools - -MCP servers expose tools that Claude can call. The GitHub MCP server provides -repository management tools, while SonarQube provides code analysis tools. -By listing their tools, you know what operations are possible. - -To try listing MCP tools: - -{{< tabs group="language" >}} -{{< tab name="TypeScript" >}} - -Create `02-list-tools.ts`: - -```typescript -import "dotenv/config"; -import { Sandbox } from "e2b"; - -async function listTools() { - console.log("Creating sandbox...\n"); - - const sbx = await Sandbox.betaCreate({ - envs: { - ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY!, - GITHUB_TOKEN: process.env.GITHUB_TOKEN!, - SONARQUBE_TOKEN: process.env.SONARQUBE_TOKEN!, - }, - mcp: { - githubOfficial: { - githubPersonalAccessToken: process.env.GITHUB_TOKEN!, - }, - sonarqube: { - org: process.env.SONARQUBE_ORG!, - token: process.env.SONARQUBE_TOKEN!, - url: "https://sonarcloud.io", - }, - }, - }); - - const mcpUrl = sbx.betaGetMcpUrl(); - const mcpToken = await sbx.betaGetMcpToken(); - - // Wait for MCP initialization - await new Promise((resolve) => setTimeout(resolve, 1000)); - - await sbx.commands.run( - `claude mcp add --transport http e2b-mcp-gateway ${mcpUrl} --header "Authorization: Bearer ${mcpToken}"`, - { timeoutMs: 0, onStdout: console.log, onStderr: console.log }, - ); - - console.log("\nDiscovering available MCP tools...\n"); - - const prompt = - "List all MCP tools you have access to. For each tool, show its exact name and a brief description."; - - await sbx.commands.run( - `echo '${prompt}' | claude -p --dangerously-skip-permissions`, - { timeoutMs: 0, onStdout: console.log, onStderr: console.log }, - ); - - await sbx.kill(); -} - -listTools().catch(console.error); -``` - -Run the script: - -```bash -npx tsx 02-list-tools.ts -``` - -{{< /tab >}} -{{< tab name="Python" >}} - -Create `02_list_tools.py`: - -```python -import os -import asyncio -from dotenv import load_dotenv -from e2b import AsyncSandbox - -load_dotenv() - -async def list_tools(): - print("Creating sandbox...\n") - - sbx = await AsyncSandbox.beta_create( - envs={ - "ANTHROPIC_API_KEY": os.getenv("ANTHROPIC_API_KEY"), - "GITHUB_TOKEN": os.getenv("GITHUB_TOKEN"), - "SONARQUBE_TOKEN": os.getenv("SONARQUBE_TOKEN"), - }, - mcp={ - "githubOfficial": { - "githubPersonalAccessToken": os.getenv("GITHUB_TOKEN"), - }, - "sonarqube": { - "org": os.getenv("SONARQUBE_ORG"), - "token": os.getenv("SONARQUBE_TOKEN"), - "url": "https://sonarcloud.io", - }, - }, - ) - - mcp_url = sbx.beta_get_mcp_url() - mcp_token = await sbx.beta_get_mcp_token() - - # Wait for MCP initialization - await asyncio.sleep(1) - - await sbx.commands.run( - f'claude mcp add --transport http e2b-mcp-gateway {mcp_url} --header "Authorization: Bearer {mcp_token}"', - timeout=0, - on_stdout=print, - on_stderr=print, - ) - - print("\nDiscovering available MCP tools...\n") - - prompt = "List all MCP tools you have access to. For each tool, show its exact name and a brief description." - - await sbx.commands.run( - f"echo '{prompt}' | claude -p --dangerously-skip-permissions", - timeout=0, - on_stdout=print, - on_stderr=print, - ) - - await sbx.kill() - -if __name__ == "__main__": - asyncio.run(list_tools()) -``` - -Run the script: - -```bash -python 02_list_tools.py -``` - -{{< /tab >}} -{{< /tabs >}} - -In the console, you should see a list of MCP tools: - -```console {collapse=true} -Creating sandbox... - -Sandbox created -Connecting to MCP gateway... - -Discovering available MCP tools... - -I have access to the following MCP tools: - -**GitHub Tools:** -1. mcp__create_repository - Create a new GitHub repository -2. mcp__list_issues - List issues in a repository -3. mcp__create_issue - Create a new issue -4. mcp__get_file_contents - Get file contents from a repository -5. mcp__create_or_update_file - Create or update files in a repository -6. mcp__create_pull_request - Create a pull request -7. mcp__create_branch - Create a new branch -8. mcp__push_files - Push multiple files in a single commit -... (30+ more GitHub tools) - -**SonarQube Tools:** -1. mcp__get_projects - List projects in organization -2. mcp__get_quality_gate_status - Get quality gate status for a project -3. mcp__list_project_issues - List quality issues in a project -4. mcp__search_issues - Search for specific quality issues -... (SonarQube analysis tools) -``` - -## Step 3: Test GitHub MCP tools - -Let's try testing GitHub using MCP tools. Start simple by listing -repository issues. - -{{< tabs group="language" >}} -{{< tab name="TypeScript" >}} - -Create `03-test-github.ts`: - -```typescript -import "dotenv/config"; -import { Sandbox } from "e2b"; - -async function testGitHub() { - console.log("Creating sandbox...\n"); - - const sbx = await Sandbox.betaCreate({ - envs: { - ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY!, - GITHUB_TOKEN: process.env.GITHUB_TOKEN!, - }, - mcp: { - githubOfficial: { - githubPersonalAccessToken: process.env.GITHUB_TOKEN!, - }, - }, - }); - - const mcpUrl = sbx.betaGetMcpUrl(); - const mcpToken = await sbx.betaGetMcpToken(); - - await new Promise((resolve) => setTimeout(resolve, 1000)); - - await sbx.commands.run( - `claude mcp add --transport http e2b-mcp-gateway ${mcpUrl} --header "Authorization: Bearer ${mcpToken}"`, - { timeoutMs: 0, onStdout: console.log, onStderr: console.log }, - ); - - const repoPath = `${process.env.GITHUB_OWNER}/${process.env.GITHUB_REPO}`; - - console.log(`\nListing issues in ${repoPath}...\n`); - - const prompt = `Using the GitHub MCP tools, list all open issues in the repository "${repoPath}". Show the issue number, title, and author for each.`; - - await sbx.commands.run( - `echo '${prompt.replace(/'/g, "'\\''")}' | claude -p --dangerously-skip-permissions`, - { - timeoutMs: 0, - onStdout: console.log, - onStderr: console.log, - }, - ); - - await sbx.kill(); -} - -testGitHub().catch(console.error); -``` - -Run the script: - -```bash -npx tsx 03-test-github.ts -``` - -{{< /tab >}} -{{< tab name="Python" >}} - -Create `03_test_github.py`: - -```python -import os -import asyncio -from dotenv import load_dotenv -from e2b import AsyncSandbox - -load_dotenv() - -async def test_github(): - print("Creating sandbox...\n") - - sbx = await AsyncSandbox.beta_create( - envs={ - "ANTHROPIC_API_KEY": os.getenv("ANTHROPIC_API_KEY"), - "GITHUB_TOKEN": os.getenv("GITHUB_TOKEN"), - }, - mcp={ - "githubOfficial": { - "githubPersonalAccessToken": os.getenv("GITHUB_TOKEN"), - }, - }, - ) - - mcp_url = sbx.beta_get_mcp_url() - mcp_token = await sbx.beta_get_mcp_token() - - await asyncio.sleep(1) - - await sbx.commands.run( - f'claude mcp add --transport http e2b-mcp-gateway {mcp_url} --header "Authorization: Bearer {mcp_token}"', - timeout=0, - on_stdout=print, - on_stderr=print, - ) - - repo_path = f"{os.getenv('GITHUB_OWNER')}/{os.getenv('GITHUB_REPO')}" - - print(f"\nListing issues in {repo_path}...\n") - - prompt = f'Using the GitHub MCP tools, list all open issues in the repository "{repo_path}". Show the issue number, title, and author for each.' - - await sbx.commands.run( - f"echo '{prompt}' | claude -p --dangerously-skip-permissions", - timeout=0, - on_stdout=print, - on_stderr=print, - ) - - await sbx.kill() - -if __name__ == "__main__": - asyncio.run(test_github()) -``` - -Run the script: - -```bash -python 03_test_github.py -``` - -{{< /tab >}} -{{< /tabs >}} - -You should see Claude use the GitHub MCP tools to list your repository's issues: - -```console {collapse=true} -Creating sandbox... -Connecting to MCP gateway... - -Listing issues in ... - -Here are the first 10 open issues in the repository: - -1. **Issue #23577**: Update README (author: user1) -2. **Issue #23575**: release-notes for Compose v2.40.1 version (author: user2) -3. **Issue #23570**: engine-cli: fix `docker volume prune` output (author: user3) -4. **Issue #23568**: Engdocs update (author: user4) -5. **Issue #23565**: add new section (author: user5) -... (continues with more issues) -``` - -You can now send prompts to Claude and interact with GitHub through -natural language. Claude decides what tool to call based on your prompt. - -## Step 4: Test SonarQube MCP tools - -Let's analyze code quality using SonarQube MCP tools. - -{{< tabs group="language" >}} -{{< tab name="TypeScript" >}} - -Create `04-test-sonarqube.ts`: - -```typescript -import "dotenv/config"; -import { Sandbox } from "e2b"; - -async function testSonarQube() { - console.log("Creating sandbox...\n"); - - const sbx = await Sandbox.betaCreate({ - envs: { - ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY!, - GITHUB_TOKEN: process.env.GITHUB_TOKEN!, - SONARQUBE_TOKEN: process.env.SONARQUBE_TOKEN!, - }, - mcp: { - githubOfficial: { - githubPersonalAccessToken: process.env.GITHUB_TOKEN!, - }, - sonarqube: { - org: process.env.SONARQUBE_ORG!, - token: process.env.SONARQUBE_TOKEN!, - url: "https://sonarcloud.io", - }, - }, - }); - - const mcpUrl = sbx.betaGetMcpUrl(); - const mcpToken = await sbx.betaGetMcpToken(); - - await new Promise((resolve) => setTimeout(resolve, 1000)); - - await sbx.commands.run( - `claude mcp add --transport http e2b-mcp-gateway ${mcpUrl} --header "Authorization: Bearer ${mcpToken}"`, - { timeoutMs: 0, onStdout: console.log, onStderr: console.log }, - ); - - console.log("\nAnalyzing code quality with SonarQube...\n"); - - const prompt = `Using the SonarQube MCP tools: - 1. List all projects in my organization - 2. For the first project, show: - - Quality gate status (pass/fail) - - Number of bugs - - Number of code smells - - Number of security vulnerabilities - 3. List the top 5 most critical issues found`; - - await sbx.commands.run( - `echo '${prompt.replace(/'/g, "'\\''")}' | claude -p --dangerously-skip-permissions`, - { - timeoutMs: 0, - onStdout: console.log, - onStderr: console.log, - }, - ); - - await sbx.kill(); -} - -testSonarQube().catch(console.error); -``` - -Run the script: - -```bash -npx tsx 04-test-sonarqube.ts -``` - -{{< /tab >}} -{{< tab name="Python" >}} - -Create `04_test_sonarqube.py`: - -```python -import os -import asyncio -from dotenv import load_dotenv -from e2b import AsyncSandbox - -load_dotenv() - -async def test_sonarqube(): - print("Creating sandbox...\n") - - sbx = await AsyncSandbox.beta_create( - envs={ - "ANTHROPIC_API_KEY": os.getenv("ANTHROPIC_API_KEY"), - "GITHUB_TOKEN": os.getenv("GITHUB_TOKEN"), - "SONARQUBE_TOKEN": os.getenv("SONARQUBE_TOKEN"), - }, - mcp={ - "githubOfficial": { - "githubPersonalAccessToken": os.getenv("GITHUB_TOKEN"), - }, - "sonarqube": { - "org": os.getenv("SONARQUBE_ORG"), - "token": os.getenv("SONARQUBE_TOKEN"), - "url": "https://sonarcloud.io", - }, - }, - ) - - mcp_url = sbx.beta_get_mcp_url() - mcp_token = await sbx.beta_get_mcp_token() - - await asyncio.sleep(1) - - await sbx.commands.run( - f'claude mcp add --transport http e2b-mcp-gateway {mcp_url} --header "Authorization: Bearer {mcp_token}"', - timeout=0, - on_stdout=print, - on_stderr=print, - ) - - print("\nAnalyzing code quality with SonarQube...\n") - - prompt = """Using the SonarQube MCP tools: - 1. List all projects in my organization - 2. For the first project, show: - - Quality gate status (pass/fail) - - Number of bugs - - Number of code smells - - Number of security vulnerabilities - 3. List the top 5 most critical issues found""" - - await sbx.commands.run( - f"echo '{prompt}' | claude -p --dangerously-skip-permissions", - timeout=0, - on_stdout=print, - on_stderr=print, - ) - - await sbx.kill() - -if __name__ == "__main__": - asyncio.run(test_sonarqube()) -``` - -Run the script: - -```bash -python 04_test_sonarqube.py -``` - -{{< /tab >}} -{{< /tabs >}} - -> [!NOTE] -> -> This script may take a few minutes to run. - -You should see Claude output SonarQube analysis results: - -```console {collapse=true} -Creating sandbox... - -Analyzing code quality with SonarQube... - -## SonarQube Analysis Results - -### 1. Projects in Your Organization - -Found **1 project**: -- **Project Name**: project-1 -- **Project Key**: project-testing - -### 2. Project Analysis - -... - -### 3. Top 5 Most Critical Issues - -Found 1 total issues (all are code smells with no critical/blocker severity): - -1. **MAJOR Severity** - test.js:2 - - **Rule**: javascript:S1854 - - **Message**: Remove this useless assignment to variable "unusedVariable" - - **Status**: OPEN - -**Summary**: The project is in good health with no bugs or vulnerabilities detected. -``` - -You can now use SonarQube MCP tools to analyze code quality through -natural language. You can retrieve quality metrics, identify issues, -and understand what code needs fixing. - -## Step 5: Create a branch and make code changes - -Now, let's teach Claude to fix code based on quality issues discovered -by SonarQube. - -{{< tabs group="language" >}} -{{< tab name="TypeScript" >}} - -Create `05-fix-code-issue.ts`: - -```typescript -import "dotenv/config"; -import { Sandbox } from "e2b"; - -async function fixCodeIssue() { - console.log("Creating sandbox...\n"); - - const sbx = await Sandbox.betaCreate({ - envs: { - ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY!, - GITHUB_TOKEN: process.env.GITHUB_TOKEN!, - SONARQUBE_TOKEN: process.env.SONARQUBE_TOKEN!, - }, - mcp: { - githubOfficial: { - githubPersonalAccessToken: process.env.GITHUB_TOKEN!, - }, - sonarqube: { - org: process.env.SONARQUBE_ORG!, - token: process.env.SONARQUBE_TOKEN!, - url: "https://sonarcloud.io", - }, - }, - }); - - const mcpUrl = sbx.betaGetMcpUrl(); - const mcpToken = await sbx.betaGetMcpToken(); - - await new Promise((resolve) => setTimeout(resolve, 1000)); - - await sbx.commands.run( - `claude mcp add --transport http e2b-mcp-gateway ${mcpUrl} --header "Authorization: Bearer ${mcpToken}"`, - { timeoutMs: 0, onStdout: console.log, onStderr: console.log }, - ); - - const repoPath = `${process.env.GITHUB_OWNER}/${process.env.GITHUB_REPO}`; - const branchName = `quality-fix-${Date.now()}`; - - console.log("\nFixing a code quality issue...\n"); - - const prompt = `Using GitHub and SonarQube MCP tools: - - 1. Analyze code quality in repository "${repoPath}" with SonarQube - 2. Find ONE simple issue that can be confidently fixed (like an unused variable or code smell) - 3. Create a new branch called "${branchName}" - 4. Read the file containing the issue using GitHub tools - 5. Fix the issue in the code - 6. Commit the fix to the new branch with a clear commit message - - Important: Only fix issues you're 100% confident about. Explain what you're fixing and why.`; - - await sbx.commands.run( - `echo '${prompt.replace(/'/g, "'\\''")}' | claude -p --dangerously-skip-permissions`, - { - timeoutMs: 0, - onStdout: console.log, - onStderr: console.log, - }, - ); - - console.log(`\nœCheck your repository for branch: ${branchName}`); - - await sbx.kill(); -} - -fixCodeIssue().catch(console.error); -``` - -Run the script: - -```bash -npx tsx 05-fix-code-issue.ts -``` - -{{< /tab >}} -{{< tab name="Python" >}} - -Create `05_fix_code_issue.py`: - -```python -import os -import asyncio -import time -from dotenv import load_dotenv -from e2b import AsyncSandbox - -load_dotenv() - -async def fix_code_issue(): - print("Creating sandbox...\n") - - sbx = await AsyncSandbox.beta_create( - envs={ - "ANTHROPIC_API_KEY": os.getenv("ANTHROPIC_API_KEY"), - "GITHUB_TOKEN": os.getenv("GITHUB_TOKEN"), - "SONARQUBE_TOKEN": os.getenv("SONARQUBE_TOKEN"), - }, - mcp={ - "githubOfficial": { - "githubPersonalAccessToken": os.getenv("GITHUB_TOKEN"), - }, - "sonarqube": { - "org": os.getenv("SONARQUBE_ORG"), - "token": os.getenv("SONARQUBE_TOKEN"), - "url": "https://sonarcloud.io", - }, - }, - ) - - mcp_url = sbx.beta_get_mcp_url() - mcp_token = await sbx.beta_get_mcp_token() - - await asyncio.sleep(1) - - await sbx.commands.run( - f'claude mcp add --transport http e2b-mcp-gateway {mcp_url} --header "Authorization: Bearer {mcp_token}"', - timeout=0, - on_stdout=print, - on_stderr=print, - ) - - repo_path = f"{os.getenv('GITHUB_OWNER')}/{os.getenv('GITHUB_REPO')}" - branch_name = f"quality-fix-{int(time.time() * 1000)}" - - print("\nFixing a code quality issue...\n") - - prompt = f"""Using GitHub and SonarQube MCP tools: - - 1. Analyze code quality in repository "{repo_path}" with SonarQube - 2. Find ONE simple issue that can be confidently fixed (like an unused variable or code smell) - 3. Create a new branch called "{branch_name}" - 4. Read the file containing the issue using GitHub tools - 5. Fix the issue in the code - 6. Commit the fix to the new branch with a clear commit message - - Important: Only fix issues you're 100% confident about. Explain what you're fixing and why.""" - - await sbx.commands.run( - f"echo '{prompt}' | claude -p --dangerously-skip-permissions", - timeout=0, - on_stdout=print, - on_stderr=print, - ) - - print(f"\n Check your repository for branch: {branch_name}") - - await sbx.kill() - -if __name__ == "__main__": - asyncio.run(fix_code_issue()) -``` - -Run the script: - -```bash -python 05_fix_code_issue.py -``` - -{{< /tab >}} -{{< /tabs >}} - -> [!NOTE] -> -> This script may take a few minutes to run. - -Claude will analyze your repository and fix a code quality issue: - -```console {collapse=true} -Creating sandbox... - -Fixing a code quality issue... - -## Task Completed Successfully! - -I've successfully analyzed the repository and fixed a code quality issue. Here's what I did: - -### 1. Analysis -Found 1 SonarQube issues in the `` repository: -- **Unused variable** (MINOR severity, line 2) -- Useless assignment (MAJOR severity, line 2) -- For-of loop suggestion (MINOR severity, lines 5-7) - -### 2. Issue Fixed -**Location:** test.js:2 -**Issue:** Unused variable declaration -**SonarQube Rule:** javascript:S1481 -**What was wrong:** The line `const unusedVariable = "this is not used";` declared a variable that was never used anywhere in the function. - -### 3. Why This Fix is Safe -- The variable was only declared and assigned, never referenced -- Removing it has zero impact on functionality -- It eliminates dead code and improves code quality -- This is a textbook example of a code smell that should be removed - -### 4. Changes Made -- Created branch: `quality-fix-xxxxx` -- Removed the unused variable declaration -- Committed with a clear message explaining the fix and referencing the SonarQube rule - -**Commit SHA:** `xxxxxxxxxxxxxxxxxxxxxxxxx` -**Branch:** https://github.com///tree/quality-fix-xxxxx - -The fix is ready for review and can be merged to improve the code quality of the repository! -``` - -You can now use GitHub and SonarQube MCP tools in the same workflow to read -files, make code changes, and commit them. - -## Step 6: Create quality-gated pull requests - -Finally, let's build the complete workflow: analyze quality, fix issues, -and create a PR only if improvements are made. - -{{< tabs group="language" >}} -{{< tab name="TypeScript" >}} - -Create `06-quality-gated-pr.ts`: - -```typescript -import "dotenv/config"; -import { Sandbox } from "e2b"; - -async function qualityGatedPR() { - console.log("Creating sandbox for quality-gated PR workflow...\n"); - - const sbx = await Sandbox.betaCreate({ - envs: { - ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY!, - GITHUB_TOKEN: process.env.GITHUB_TOKEN!, - SONARQUBE_TOKEN: process.env.SONARQUBE_TOKEN!, - }, - mcp: { - githubOfficial: { - githubPersonalAccessToken: process.env.GITHUB_TOKEN!, - }, - sonarqube: { - org: process.env.SONARQUBE_ORG!, - token: process.env.SONARQUBE_TOKEN!, - url: "https://sonarcloud.io", - }, - }, - }); - - const mcpUrl = sbx.betaGetMcpUrl(); - const mcpToken = await sbx.betaGetMcpToken(); - - await new Promise((resolve) => setTimeout(resolve, 1000)); - - await sbx.commands.run( - `claude mcp add --transport http e2b-mcp-gateway ${mcpUrl} --header "Authorization: Bearer ${mcpToken}"`, - { timeoutMs: 0, onStdout: console.log, onStderr: console.log }, - ); - - const repoPath = `${process.env.GITHUB_OWNER}/${process.env.GITHUB_REPO}`; - const branchName = `quality-improvements-${Date.now()}`; - - console.log("\nRunning quality-gated PR workflow...\n"); - - const prompt = `You are a code quality engineer. Using GitHub and SonarQube MCP tools: - - STEP 1: ANALYSIS - - Get current code quality status from SonarQube for "${repoPath}" - - Record the current number of bugs, code smells, and vulnerabilities - - Identify 1-3 issues that you can confidently fix - - STEP 2: FIX ISSUES - - Create branch "${branchName}" - - For each issue you're fixing: - * Read the file with the issue - * Make the fix - * Commit with a descriptive message - - Only fix issues where you're 100% confident the fix is correct - - STEP 3: VERIFICATION - - After your fixes, check if quality metrics would improve - - Calculate: Would this reduce bugs/smells/vulnerabilities? - - STEP 4: QUALITY GATE - - Only proceed if your changes improve quality - - If quality would not improve, explain why and stop - - STEP 5: CREATE PR (only if quality gate passes) - - Create a pull request from "${branchName}" to main - - Title: "Quality improvements: [describe what you fixed]" - - Description should include: - * What issues you fixed - * Before/after quality metrics - * Why these fixes improve code quality - - Add a comment with detailed SonarQube analysis - - Be thorough and explain your decisions at each step.`; - - await sbx.commands.run( - `echo '${prompt.replace(/'/g, "'\\''")}' | claude -p --dangerously-skip-permissions`, - { - timeoutMs: 0, - onStdout: console.log, - onStderr: console.log, - }, - ); - - console.log(`\n Workflow complete! Check ${repoPath} for new pull request.`); - - await sbx.kill(); -} - -qualityGatedPR().catch(console.error); -``` - -Run the script: - -```bash -npx tsx 06-quality-gated-pr.ts -``` - -{{< /tab >}} -{{< tab name="Python" >}} - -Create `06_quality_gated_pr.py`: - -```python -import os -import asyncio -import time -from dotenv import load_dotenv -from e2b import AsyncSandbox - -load_dotenv() - -async def quality_gated_pr(): - print("Creating sandbox for quality-gated PR workflow...\n") - - sbx = await AsyncSandbox.beta_create( - envs={ - "ANTHROPIC_API_KEY": os.getenv("ANTHROPIC_API_KEY"), - "GITHUB_TOKEN": os.getenv("GITHUB_TOKEN"), - "SONARQUBE_TOKEN": os.getenv("SONARQUBE_TOKEN"), - }, - mcp={ - "githubOfficial": { - "githubPersonalAccessToken": os.getenv("GITHUB_TOKEN"), - }, - "sonarqube": { - "org": os.getenv("SONARQUBE_ORG"), - "token": os.getenv("SONARQUBE_TOKEN"), - "url": "https://sonarcloud.io", - }, - }, - ) - - mcp_url = sbx.beta_get_mcp_url() - mcp_token = await sbx.beta_get_mcp_token() - - await asyncio.sleep(1) - - await sbx.commands.run( - f'claude mcp add --transport http e2b-mcp-gateway {mcp_url} --header "Authorization: Bearer {mcp_token}"', - timeout=0, - on_stdout=print, - on_stderr=print, - ) - - repo_path = f"{os.getenv('GITHUB_OWNER')}/{os.getenv('GITHUB_REPO')}" - branch_name = f"quality-improvements-{int(time.time() * 1000)}" - - print("\nRunning quality-gated PR workflow...\n") - - prompt = f"""You are a code quality engineer. Using GitHub and SonarQube MCP tools: - - STEP 1: ANALYSIS - - Get current code quality status from SonarQube for "{repo_path}" - - Record the current number of bugs, code smells, and vulnerabilities - - Identify 1-3 issues that you can confidently fix - - STEP 2: FIX ISSUES - - Create branch "{branch_name}" - - For each issue you are fixing: - Read the file with the issue - Make the fix - Commit with a descriptive message - - Only fix issues where you are 100 percent confident the fix is correct - - STEP 3: VERIFICATION - - After your fixes, check if quality metrics would improve - - Calculate: Would this reduce bugs/smells/vulnerabilities? - - STEP 4: QUALITY GATE - - Only proceed if your changes improve quality - - If quality would not improve, explain why and stop - - STEP 5: CREATE PR (only if quality gate passes) - - Create a pull request from "{branch_name}" to main - - Title: "Quality improvements: [describe what you fixed]" - - Description should include: - What issues you fixed - Before/after quality metrics - Why these fixes improve code quality - - Add a comment with detailed SonarQube analysis - - Be thorough and explain your decisions at each step.""" - - await sbx.commands.run( - f"echo '{prompt.replace(chr(39), chr(39) + chr(92) + chr(39) + chr(39))}' | claude -p --dangerously-skip-permissions", - timeout=0, - on_stdout=print, - on_stderr=print, - ) - - print(f"\n Workflow complete! Check {repo_path} for new pull request.") - - await sbx.kill() - -if __name__ == "__main__": - asyncio.run(quality_gated_pr()) -``` - -Run the script: - -```bash -python 06_quality_gated_pr.py -``` - -{{< /tab >}} -{{< /tabs >}} - -> [!NOTE] -> -> This script may take a few minutes to run. - -Claude will run the entire workflow, creating a quality improvement -and opening a PR in GitHub: - -```console {collapse=true} -Creating sandbox for quality-gated PR workflow... - -Running quality-gated PR workflow... - -## Complete - Code Quality Improvement Successfully Executed - -### Summary of Work Completed - -**Pull Request Created:** https://github.com///pull/1 - ---- - -### STEP 1: ANALYSIS - -**Current State:** -- **Bugs:** 0 -- **Vulnerabilities:** 0 -- **Code Smells:** 1 -- **Lines of Code:** 12 -- **All Ratings:** A (Reliability, Security, Maintainability) - -**Issues Identified (all in test.js):** -1. xxxxxxxxx - Unused variable declaration (MINOR) - ---- - -### STEP 2: FIX ISSUES - -**Branch Created:** `quality-improvements-xxxxxxxx` - -**Changes Made:** -1. **Removed unused variable** (line 2) - Eliminated dead code that served no purpose -2. **Modernized loop pattern** (lines 5-7) - Converted `for (let i = 0; i < items.length; i++)` to `for (const item of items)` - -**Commit:** xxxxxxxxxx - ---- - -### STEP 3: VERIFICATION - -**Expected Impact:** -- Code Smells: 1 → 0 (100% reduction) -- Bugs: 0 → 0 (maintained) -- Vulnerabilities: 0 → 0 (maintained) -- All quality ratings maintained at A - ---- - -### STEP 4: QUALITY GATE PASSED - -**Decision Criteria Met:** -- ✅ Reduces code smells by 100% -- ✅ No new bugs or vulnerabilities introduced -- ✅ Code is more readable and maintainable -- ✅ Follows modern JavaScript best practices -- ✅ All fixes are low-risk refactorings with no behavioral changes - ---- - -### STEP 5: CREATE PR - -**Pull Request Details:** -- **Number:** #1 -- **Title:** Quality improvements: Remove unused variable and modernize for loop -- **Branch:** quality-improvements-xxxxxxxx → main -- **URL:** https://github.com//pull/1 - -**PR Includes:** -- Comprehensive description with before/after metrics -- Detailed SonarQube analysis comment with issue breakdown -- Code comparison showing improvements -- Quality metrics table - -The pull request is now ready for review and merge! -``` - -You've now built a complete, multi-step workflow with conditional logic. -Claude analyzes quality with SonarQube, makes fixes using GitHub tools, -verifies improvements, and only creates a PR if quality actually improves. - -## Step 7: Add error handling - -Production workflows need error handling. Let's make the workflow more robust. - -{{< tabs group="language" >}} -{{< tab name="TypeScript" >}} - -Create `07-robust-workflow.ts`: - -```typescript -import "dotenv/config"; -import { Sandbox } from "e2b"; - -async function robustWorkflow() { - let sbx: Sandbox | undefined; - - try { - console.log("Creating sandbox...\n"); - - sbx = await Sandbox.betaCreate({ - envs: { - ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY!, - GITHUB_TOKEN: process.env.GITHUB_TOKEN!, - SONARQUBE_TOKEN: process.env.SONARQUBE_TOKEN!, - }, - mcp: { - githubOfficial: { - githubPersonalAccessToken: process.env.GITHUB_TOKEN!, - }, - sonarqube: { - org: process.env.SONARQUBE_ORG!, - token: process.env.SONARQUBE_TOKEN!, - url: "https://sonarcloud.io", - }, - }, - }); - - const mcpUrl = sbx.betaGetMcpUrl(); - const mcpToken = await sbx.betaGetMcpToken(); - - await new Promise((resolve) => setTimeout(resolve, 1000)); - - await sbx.commands.run( - `claude mcp add --transport http e2b-mcp-gateway ${mcpUrl} --header "Authorization: Bearer ${mcpToken}"`, - { timeoutMs: 0, onStdout: console.log, onStderr: console.log }, - ); - - const repoPath = `${process.env.GITHUB_OWNER}/${process.env.GITHUB_REPO}`; - - console.log("\nRunning workflow with error handling...\n"); - - const prompt = `Run a quality improvement workflow for "${repoPath}". - - ERROR HANDLING RULES: - 1. If SonarQube is unreachable, explain the error and stop gracefully - 2. If GitHub API fails, retry once, then explain and stop - 3. If no fixable issues are found, explain why and exit (this is not an error) - 4. If file modifications fail, explain which file and why - 5. At each step, check for errors before proceeding - - Run the workflow and handle any errors you encounter professionally.`; - - await sbx.commands.run( - `echo '${prompt.replace(/'/g, "'\\''")}' | claude -p --dangerously-skip-permissions`, - { - timeoutMs: 0, - onStdout: console.log, - onStderr: console.log, - }, - ); - - console.log("\n Workflow completed"); - } catch (error) { - const err = error as Error; - console.error("\n Workflow failed:", err.message); - - if (err.message.includes("403")) { - console.error("\n Check your E2B account has MCP gateway access"); - } else if (err.message.includes("401")) { - console.error("\n Check your API tokens are valid"); - } else if (err.message.includes("Credit balance")) { - console.error("\n Check your Anthropic API credit balance"); - } - - process.exit(1); - } finally { - if (sbx) { - console.log("\n Cleaning up sandbox..."); - await sbx.kill(); - } - } -} - -robustWorkflow().catch(console.error); -``` - -Run the script: - -```bash -npx tsx 07-robust-workflow.ts -``` - -{{< /tab >}} -{{< tab name="Python" >}} - -Create `07_robust_workflow.py`: - -```python -import os -import asyncio -import sys -from dotenv import load_dotenv -from e2b import AsyncSandbox - -load_dotenv() - -async def robust_workflow(): - sbx = None - - try: - print("Creating sandbox...\n") - - sbx = await AsyncSandbox.beta_create( - envs={ - "ANTHROPIC_API_KEY": os.getenv("ANTHROPIC_API_KEY"), - "GITHUB_TOKEN": os.getenv("GITHUB_TOKEN"), - "SONARQUBE_TOKEN": os.getenv("SONARQUBE_TOKEN"), - }, - mcp={ - "githubOfficial": { - "githubPersonalAccessToken": os.getenv("GITHUB_TOKEN"), - }, - "sonarqube": { - "org": os.getenv("SONARQUBE_ORG"), - "token": os.getenv("SONARQUBE_TOKEN"), - "url": "https://sonarcloud.io", - }, - }, - ) - - mcp_url = sbx.beta_get_mcp_url() - mcp_token = await sbx.beta_get_mcp_token() - - await asyncio.sleep(1) - - await sbx.commands.run( - f'claude mcp add --transport http e2b-mcp-gateway {mcp_url} --header "Authorization: Bearer {mcp_token}"', - timeout=0, # Fixed: was timeout_ms - on_stdout=print, - on_stderr=print, - ) - - repo_path = f"{os.getenv('GITHUB_OWNER')}/{os.getenv('GITHUB_REPO')}" - - print("\nRunning workflow with error handling...\n") - - prompt = f"""Run a quality improvement workflow for "{repo_path}". - - ERROR HANDLING RULES: - 1. If SonarQube is unreachable, explain the error and stop gracefully - 2. If GitHub API fails, retry once, then explain and stop - 3. If no fixable issues are found, explain why and exit (this is not an error) - 4. If file modifications fail, explain which file and why - 5. At each step, check for errors before proceeding - - Run the workflow and handle any errors you encounter professionally.""" - - await sbx.commands.run( - f"echo '{prompt}' | claude -p --dangerously-skip-permissions", - timeout=0, - on_stdout=print, - on_stderr=print, - ) - - print("\n Workflow completed") - - except Exception as error: - print(f"\n✗ Workflow failed: {str(error)}") - - error_msg = str(error) - if "403" in error_msg: - print("\n Check your E2B account has MCP gateway access") - elif "401" in error_msg: - print("\n Check your API tokens are valid") - elif "Credit balance" in error_msg: - print("\n Check your Anthropic API credit balance") - - sys.exit(1) - - finally: - if sbx: - print("\n Cleaning up sandbox...") - await sbx.kill() - -if __name__ == "__main__": - asyncio.run(robust_workflow()) -``` - -Run the script: - -```bash -python 07_robust_workflow.py -``` - -{{< /tab >}} -{{< /tabs >}} - -Claude will run the entire workflow, and if it encounters an error, respond -with robust error messaging. - -## Next steps - -In the next section, you'll customize your workflow for your needs. diff --git a/content/guides/go-prometheus-monitoring/_index.md b/content/guides/go-prometheus-monitoring/_index.md index a295ab77a144..78e506a5085b 100644 --- a/content/guides/go-prometheus-monitoring/_index.md +++ b/content/guides/go-prometheus-monitoring/_index.md @@ -5,11 +5,17 @@ title: Monitor a Golang application with Prometheus and Grafana summary: | Learn how to containerize a Golang application and monitor it with Prometheus and Grafana. linkTitle: Prometheus and Grafana -languages: [go] +aliases: + - /guides/go-prometheus-monitoring/application/ + - /guides/go-prometheus-monitoring/compose/ + - /guides/go-prometheus-monitoring/containerize/ + - /guides/go-prometheus-monitoring/develop/ params: + tags: [cicd] time: 45 minutes --- + The guide teaches you how to containerize a Golang application and monitor it with Prometheus and Grafana. > **Acknowledgment** @@ -38,3 +44,586 @@ In this guide, you will be creating a Golang server with some endpoints to simul ## Next steps You will create a Golang server and expose metrics using Prometheus. + +## Building the application + +### Prerequisites + +* You have a [Git client](https://git-scm.com/downloads). The examples in this section use a command-line based Git client, but you can use any client. + +You will be creating a Golang server with some endpoints to simulate a real-world application. Then you will expose metrics from the server using Prometheus. + +### Getting the sample application + +Clone the sample application to use with this guide. Open a terminal, change +directory to a directory that you want to work in, and run the following +command to clone the repository: + +```console +$ git clone https://github.com/dockersamples/go-prometheus-monitoring.git +``` + +Once you cloned you will see the following content structure inside `go-prometheus-monitoring` directory, + +```text +go-prometheus-monitoring +├── CONTRIBUTING.md +├── Docker +│ ├── grafana.yml +│ └── prometheus.yml +├── dashboard.json +├── Dockerfile +├── LICENSE +├── README.md +├── compose.yaml +├── go.mod +├── go.sum +└── main.go +``` + +- **main.go** - The entry point of the application. +- **go.mod and go.sum** - Go module files. +- **Dockerfile** - Dockerfile used to build the app. +- **Docker/** - Contains the Docker Compose configuration files for Grafana and Prometheus. +- **compose.yaml** - Compose file to launch everything (Golang app, Prometheus, and Grafana). +- **dashboard.json** - Grafana dashboard configuration file. +- **Dockerfile** - Dockerfile used to build the Golang app. +- **compose.yaml** - Docker Compose file to launch everything (Golang app, Prometheus, and Grafana). +- Other files are for licensing and documentation purposes. + +### Understanding the application + +The following is the complete logic of the application you will find in `main.go`. + +```go +package main + +import ( + "strconv" + + "github.com/gin-gonic/gin" + "github.com/prometheus/client_golang/prometheus" + "github.com/prometheus/client_golang/prometheus/promhttp" +) + +// Define metrics +var ( + HttpRequestTotal = prometheus.NewCounterVec(prometheus.CounterOpts{ + Name: "api_http_request_total", + Help: "Total number of requests processed by the API", + }, []string{"path", "status"}) + + HttpRequestErrorTotal = prometheus.NewCounterVec(prometheus.CounterOpts{ + Name: "api_http_request_error_total", + Help: "Total number of errors returned by the API", + }, []string{"path", "status"}) +) + +// Custom registry (without default Go metrics) +var customRegistry = prometheus.NewRegistry() + +// Register metrics with custom registry +func init() { + customRegistry.MustRegister(HttpRequestTotal, HttpRequestErrorTotal) +} + +func main() { + router := gin.Default() + + // Register /metrics before middleware + router.GET("/metrics", PrometheusHandler()) + + router.Use(RequestMetricsMiddleware()) + router.GET("/health", func(c *gin.Context) { + c.JSON(200, gin.H{ + "message": "Up and running!", + }) + }) + router.GET("/v1/users", func(c *gin.Context) { + c.JSON(200, gin.H{ + "message": "Hello from /v1/users", + }) + }) + + router.Run(":8000") +} + +// Custom metrics handler with custom registry +func PrometheusHandler() gin.HandlerFunc { + h := promhttp.HandlerFor(customRegistry, promhttp.HandlerOpts{}) + return func(c *gin.Context) { + h.ServeHTTP(c.Writer, c.Request) + } +} + +// Middleware to record incoming requests metrics +func RequestMetricsMiddleware() gin.HandlerFunc { + return func(c *gin.Context) { + path := c.Request.URL.Path + c.Next() + status := c.Writer.Status() + if status < 400 { + HttpRequestTotal.WithLabelValues(path, strconv.Itoa(status)).Inc() + } else { + HttpRequestErrorTotal.WithLabelValues(path, strconv.Itoa(status)).Inc() + } + } +} +``` + +In this part of the code, you have imported the required packages `gin`, `prometheus`, and `promhttp`. Then you have defined a couple of variables, `HttpRequestTotal` and `HttpRequestErrorTotal` are Prometheus counter metrics, and `customRegistry` is a custom registry that will be used to register these metrics. The name of the metric is a string that you can use to identify the metric. The help string is a string that will be shown when you query the `/metrics` endpoint to understand the metric. The reason you are using the custom registry is so avoid the default Go metrics that are registered by default by the Prometheus client. Then using the `init` function you are registering the metrics with the custom registry. + +```go +import ( + "strconv" + + "github.com/gin-gonic/gin" + "github.com/prometheus/client_golang/prometheus" + "github.com/prometheus/client_golang/prometheus/promhttp" +) + +// Define metrics +var ( + HttpRequestTotal = prometheus.NewCounterVec(prometheus.CounterOpts{ + Name: "api_http_request_total", + Help: "Total number of requests processed by the API", + }, []string{"path", "status"}) + + HttpRequestErrorTotal = prometheus.NewCounterVec(prometheus.CounterOpts{ + Name: "api_http_request_error_total", + Help: "Total number of errors returned by the API", + }, []string{"path", "status"}) +) + +// Custom registry (without default Go metrics) +var customRegistry = prometheus.NewRegistry() + +// Register metrics with custom registry +func init() { + customRegistry.MustRegister(HttpRequestTotal, HttpRequestErrorTotal) +} +``` + +In the `main` function, you have created a new instance of the `gin` framework and created three routes. You can see the health endpoint that is on path `/health` that will return a JSON with `{"message": "Up and running!"}` and the `/v1/users` endpoint that will return a JSON with `{"message": "Hello from /v1/users"}`. The third route is for the `/metrics` endpoint that will return the metrics in the Prometheus format. Then you have `RequestMetricsMiddleware` middleware, it will be called for every request made to the API. It will record the incoming requests metrics like status codes and paths. Finally, you are running the gin application on port 8000. + +```golang +func main() { + router := gin.Default() + + // Register /metrics before middleware + router.GET("/metrics", PrometheusHandler()) + + router.Use(RequestMetricsMiddleware()) + router.GET("/health", func(c *gin.Context) { + c.JSON(200, gin.H{ + "message": "Up and running!", + }) + }) + router.GET("/v1/users", func(c *gin.Context) { + c.JSON(200, gin.H{ + "message": "Hello from /v1/users", + }) + }) + + router.Run(":8000") +} +``` + +Now comes the middleware function `RequestMetricsMiddleware`. This function is called for every request made to the API. It increments the `HttpRequestTotal` counter (different counter for different paths and status codes) if the status code is less than or equal to 400. If the status code is greater than 400, it increments the `HttpRequestErrorTotal` counter (different counter for different paths and status codes). The `PrometheusHandler` function is the custom handler that will be called for the `/metrics` endpoint. It will return the metrics in the Prometheus format. + +```golang +// Custom metrics handler with custom registry +func PrometheusHandler() gin.HandlerFunc { + h := promhttp.HandlerFor(customRegistry, promhttp.HandlerOpts{}) + return func(c *gin.Context) { + h.ServeHTTP(c.Writer, c.Request) + } +} + +// Middleware to record incoming requests metrics +func RequestMetricsMiddleware() gin.HandlerFunc { + return func(c *gin.Context) { + path := c.Request.URL.Path + c.Next() + status := c.Writer.Status() + if status < 400 { + HttpRequestTotal.WithLabelValues(path, strconv.Itoa(status)).Inc() + } else { + HttpRequestErrorTotal.WithLabelValues(path, strconv.Itoa(status)).Inc() + } + } +} +``` + +That's it, this was the complete gist of the application. Now it's time to run and test if the app is registering metrics correctly. + +### Running the application + +Make sure you are still inside `go-prometheus-monitoring` directory in the terminal, and run the following command. Install the dependencies by running `go mod tidy` and then build and run the application by running `go run main.go`. Then visit `http://localhost:8000/health` or `http://localhost:8000/v1/users`. You should see the output `{"message": "Up and running!"}` or `{"message": "Hello from /v1/users"}`. If you are able to see this then your app is successfully up and running. + + +Now, check your application's metrics by accessing the `/metrics` endpoint. +Open `http://localhost:8000/metrics` in your browser. You should see similar output to the following. + +```sh +# HELP api_http_request_error_total Total number of errors returned by the API +# TYPE api_http_request_error_total counter +api_http_request_error_total{path="/",status="404"} 1 +api_http_request_error_total{path="//v1/users",status="404"} 1 +api_http_request_error_total{path="/favicon.ico",status="404"} 1 +# HELP api_http_request_total Total number of requests processed by the API +# TYPE api_http_request_total counter +api_http_request_total{path="/health",status="200"} 2 +api_http_request_total{path="/v1/users",status="200"} 1 +``` + +In the terminal, press `ctrl` + `c` to stop the application. + +> [!Note] +> If you don't want to run the application locally, and want to run it in a Docker container, skip to next page where you create a Dockerfile and containerize the application. + +### Summary + +In this section, you learned how to create a Golang app to register metrics with Prometheus. By implementing middleware functions, you were able to increment the counters based on the request path and status codes. + +### Next steps + +In the next section, you'll learn how to containerize your application. + +## Containerize a Golang application + +Containerization helps you bundle the application and its dependencies into a single package called a container. This package can run on any platform without worrying about the environment. In this section, you will learn how to containerize a Golang application using Docker. + +To containerize a Golang application, you first need to create a Dockerfile. The Dockerfile contains instructions to build and run the application in a container. Also, when creating a Dockerfile, you can follow different sets of best practices to optimize the image size and make it more secure. + +### Creating a Dockerfile + +Create a new file named `Dockerfile` in the root directory of your Golang application. The Dockerfile contains instructions to build and run the application in a container. + +The following is a Dockerfile for a Golang application. You will also find this file in the `go-prometheus-monitoring` directory. + +```dockerfile +# Use the official Golang image as the base +FROM golang:1.24-alpine AS builder + +# Set environment variables +ENV CGO_ENABLED=0 \ + GOOS=linux \ + GOARCH=amd64 + +# Set working directory inside the container +WORKDIR /build + +# Copy go.mod and go.sum files for dependency installation +COPY go.mod go.sum ./ + +# Download dependencies +RUN go mod download + +# Copy the entire application source +COPY . . + +# Build the Go binary +RUN go build -o /app . + +# Final lightweight stage +FROM alpine:3.21 AS final + +# Copy the compiled binary from the builder stage +COPY --from=builder /app /bin/app + +# Expose the application's port +EXPOSE 8000 + +# Run the application +CMD ["bin/app"] +``` + +### Understanding the Dockerfile + +The Dockerfile consists of two stages: + +1. **Build stage**: This stage uses the official Golang image as the base and sets the necessary environment variables. It also sets the working directory inside the container, copies the `go.mod` and `go.sum` files for dependency installation, downloads the dependencies, copies the entire application source, and builds the Go binary. + + You use the `golang:1.24-alpine` image as the base image for the build stage. The `CGO_ENABLED=0` environment variable disables CGO, which is useful for building static binaries. You also set the `GOOS` and `GOARCH` environment variables to `linux` and `amd64`, respectively, to build the binary for the Linux platform. + +2. **Final stage**: This stage uses the official Alpine image as the base and copies the compiled binary from the build stage. It also exposes the application's port and runs the application. + + You use the `alpine:3.21` image as the base image for the final stage. You copy the compiled binary from the build stage to the final image. You expose the application's port using the `EXPOSE` instruction and run the application using the `CMD` instruction. + + Apart from the multi-stage build, the Dockerfile also follows best practices such as using the official images, setting the working directory, and copying only the necessary files to the final image. You can further optimize the Dockerfile by other best practices. + +### Build the Docker image and run the application + +One you have the Dockerfile, you can build the Docker image and run the application in a container. + +To build the Docker image, run the following command in the terminal: + +```console +$ docker build -t go-api:latest . +``` + +After building the image, you can run the application in a container using the following command: + +```console +$ docker run -p 8000:8000 go-api:latest +``` + +The application will start running inside the container, and you can access it at `http://localhost:8000`. You can also check the running containers using the `docker ps` command. + +```console +$ docker ps +``` + +### Summary + +In this section, you learned how to containerize a Golang application using a Dockerfile. You created a multi-stage Dockerfile to build and run the application in a container. You also learned about best practices to optimize the Docker image size and make it more secure. + +Related information: + + - [Dockerfile reference](/reference/dockerfile.md) + - [.dockerignore file](/reference/dockerfile.md#dockerignore-file) + +### Next steps + +In the next section, you will learn how to use Docker Compose to connect and run multiple services together to monitor a Golang application with Prometheus and Grafana. + +## Connecting services with Docker Compose + +Now that you have containerized the Golang application, you will use Docker Compose to connect your services together. You will connect the Golang application, Prometheus, and Grafana services together to monitor the Golang application with Prometheus and Grafana. + +### Creating a Docker Compose file + +Create a new file named `compose.yml` in the root directory of your Golang application. The Docker Compose file contains instructions to run multiple services and connect them together. + +Here is a Docker Compose file for a project that uses Golang, Prometheus, and Grafana. You will also find this file in the `go-prometheus-monitoring` directory. + +```yaml +services: + api: + container_name: go-api + build: + context: . + dockerfile: Dockerfile + image: go-api:latest + ports: + - 8000:8000 + networks: + - go-network + healthcheck: + test: ["CMD", "curl", "-f", "http://localhost:8000/health"] + interval: 30s + timeout: 10s + retries: 5 + develop: + watch: + - path: . + action: rebuild + + prometheus: + container_name: prometheus + image: prom/prometheus:v2.55.0 + volumes: + - ./Docker/prometheus.yml:/etc/prometheus/prometheus.yml + ports: + - 9090:9090 + networks: + - go-network + + grafana: + container_name: grafana + image: grafana/grafana:11.3.0 + volumes: + - ./Docker/grafana.yml:/etc/grafana/provisioning/datasources/datasource.yaml + - grafana-data:/var/lib/grafana + ports: + - 3000:3000 + networks: + - go-network + environment: + - GF_SECURITY_ADMIN_USER=admin + - GF_SECURITY_ADMIN_PASSWORD=password + +volumes: + grafana-data: + +networks: + go-network: + driver: bridge +``` + +### Understanding the Docker Compose file + +The Docker Compose file consists of three services: + +- **Golang application service**: This service builds the Golang application using the Dockerfile and runs it in a container. It exposes the application's port `8000` and connects to the `go-network` network. It also defines a health check to monitor the application's health. You have also used `healthcheck` to monitor the health of the application. The health check runs every 30 seconds and retries 5 times if the health check fails. The health check uses the `curl` command to check the `/health` endpoint of the application. Apart from the health check, you have also added a `develop` section to watch the changes in the application's source code and rebuild the application using the Docker Compose Watch feature. + +- **Prometheus service**: This service runs the Prometheus server in a container. It uses the official Prometheus image `prom/prometheus:v2.55.0`. It exposes the Prometheus server on port `9090` and connects to the `go-network` network. You have also mounted the `prometheus.yml` file from the `Docker` directory which is present in the root directory of your project. The `prometheus.yml` file contains the Prometheus configuration to scrape the metrics from the Golang application. This is how you connect the Prometheus server to the Golang application. + + ```yaml + global: + scrape_interval: 10s + evaluation_interval: 10s + + scrape_configs: + - job_name: myapp + static_configs: + - targets: ["api:8000"] + ``` + + In the `prometheus.yml` file, you have defined a job named `myapp` to scrape the metrics from the Golang application. The `targets` field specifies the target to scrape the metrics from. In this case, the target is the Golang application running on port `8000`. The `api` is the service name of the Golang application in the Docker Compose file. The Prometheus server will scrape the metrics from the Golang application every 10 seconds. + +- **Grafana service**: This service runs the Grafana server in a container. It uses the official Grafana image `grafana/grafana:11.3.0`. It exposes the Grafana server on port `3000` and connects to the `go-network` network. You have also mounted the `grafana.yml` file from the `Docker` directory which is present in the root directory of your project. The `grafana.yml` file contains the Grafana configuration to add the Prometheus data source. This is how you connect the Grafana server to the Prometheus server. In the environment variables, you have set the Grafana admin user and password, which will be used to log in to the Grafana dashboard. + + ```yaml + apiVersion: 1 + datasources: + - name: Prometheus (Main) + type: prometheus + url: http://prometheus:9090 + isDefault: true + ``` + + In the `grafana.yml` file, you have defined a Prometheus data source named `Prometheus (Main)`. The `type` field specifies the type of the data source, which is `prometheus`. The `url` field specifies the URL of the Prometheus server to fetch the metrics from. In this case, the URL is `http://prometheus:9090`. `prometheus` is the service name of the Prometheus server in the Docker Compose file. The `isDefault` field specifies whether the data source is the default data source in Grafana. + +Apart from the services, the Docker Compose file also defines a volume named `grafana-data` to persist the Grafana data and a network named `go-network` to connect the services together. You have created a custom network `go-network` to connect the services together. The `driver: bridge` field specifies the network driver to use for the network. + +### Building and running the services + +Now that you have the Docker Compose file, you can build the services and run them together using Docker Compose. + +To build and run the services, run the following command in the terminal: + +```console +$ docker compose up +``` + +The `docker compose up` command builds the services defined in the Docker Compose file and runs them together. You will see the similar output in the terminal: + +```console + ✔ Network go-prometheus-monitoring_go-network Created 0.0s + ✔ Container grafana Created 0.3s + ✔ Container go-api Created 0.2s + ✔ Container prometheus Created 0.3s +Attaching to go-api, grafana, prometheus +go-api | [GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached. +go-api | +go-api | [GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production. +go-api | - using env: export GIN_MODE=release +go-api | - using code: gin.SetMode(gin.ReleaseMode) +go-api | +go-api | [GIN-debug] GET /metrics --> main.PrometheusHandler.func1 (3 handlers) +go-api | [GIN-debug] GET /health --> main.main.func1 (4 handlers) +go-api | [GIN-debug] GET /v1/users --> main.main.func2 (4 handlers) +go-api | [GIN-debug] [WARNING] You trusted all proxies, this is NOT safe. We recommend you to set a value. +go-api | Please check https://pkg.go.dev/github.com/gin-gonic/gin#readme-don-t-trust-all-proxies for details. +go-api | [GIN-debug] Listening and serving HTTP on :8000 +prometheus | ts=2025-03-15T05:57:06.676Z caller=main.go:627 level=info msg="No time or size retention was set so using the default time retention" duration=15d +prometheus | ts=2025-03-15T05:57:06.678Z caller=main.go:671 level=info msg="Starting Prometheus Server" mode=server version="(version=2.55.0, branch=HEAD, revision=91d80252c3e528728b0f88d254dd720f6be07cb8)" +grafana | logger=settings t=2025-03-15T05:57:06.865335506Z level=info msg="Config overridden from command line" arg="default.log.mode=console" +grafana | logger=settings t=2025-03-15T05:57:06.865337131Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" +grafana | logger=ngalert.state.manager t=2025-03-15T05:57:07.088956839Z level=info msg="State +. +. +grafana | logger=plugin.angulardetectorsprovider.dynamic t=2025-03-15T05:57:07.530317298Z level=info msg="Patterns update finished" duration=440.489125ms +``` + +The services will start running, and you can access the Golang application at `http://localhost:8000`, Prometheus at `http://localhost:9090/health`, and Grafana at `http://localhost:3000`. You can also check the running containers using the `docker ps` command. + +```console +$ docker ps +``` + +### Summary + +In this section, you learned how to connect services together using Docker Compose. You created a Docker Compose file to run multiple services together and connect them using networks. You also learned how to build and run the services using Docker Compose. + +Related information: + + - [Docker Compose overview](/manuals/compose/_index.md) + - [Compose file reference](/reference/compose-file/_index.md) + +Next, you will learn how to develop the Golang application with Docker Compose and monitor it with Prometheus and Grafana. + +### Next steps + +In the next section, you will learn how to develop the Golang application with Docker. You will also learn how to use Docker Compose Watch to rebuild the image whenever you make changes to the code. Lastly, you will test the application and visualize the metrics in Grafana using Prometheus as the data source. + +## Developing your application + +In the last section, you saw how using Docker Compose, you can connect your services together. In this section, you will learn how to develop the Golang application with Docker. You will also see how to use Docker Compose Watch to rebuild the image whenever you make changes to the code. Lastly, you will test the application and visualize the metrics in Grafana using Prometheus as the data source. + +### Developing the application + +Now, if you make any changes to your Golang application locally, it needs to reflect in the container, right? To do that, one approach is use the `--build` flag in Docker Compose after making changes in the code. This will rebuild all the services which have the `build` instruction in the `compose.yml` file, in your case, the `api` service (Golang application). + +```console +docker compose up --build +``` + +But, this is not the best approach. This is not efficient. Every time you make a change in the code, you need to rebuild manually. This is not is not a good flow for development. + +The better approach is to use Docker Compose Watch. In the `compose.yml` file, under the service `api`, you have added the `develop` section. So, it's more like a hot reloading. Whenever you make changes to code (defined in `path`), it will rebuild the image (or restart depending on the action). This is how you can use it: + +```yaml {hl_lines="17-20",linenos=true} +services: + api: + container_name: go-api + build: + context: . + dockerfile: Dockerfile + image: go-api:latest + ports: + - 8000:8000 + networks: + - go-network + healthcheck: + test: ["CMD", "curl", "-f", "http://localhost:8080/health"] + interval: 30s + timeout: 10s + retries: 5 + develop: + watch: + - path: . + action: rebuild +``` + +Once you have added the `develop` section in the `compose.yml` file, you can use the following command to start the development server: + +```console +$ docker compose watch +``` + +Now, if you modify your `main.go` or any other file in the project, the `api` service will be rebuilt automatically. You will see the following output in the terminal: + +```bash +Rebuilding service(s) ["api"] after changes were detected... +[+] Building 8.1s (15/15) FINISHED docker:desktop-linux + => [api internal] load build definition from Dockerfile 0.0s + => => transferring dockerfile: 704B 0.0s + => [api internal] load metadata for docker.io/library/alpine:3.17 1.1s + . + => => exporting manifest list sha256:89ebc86fd51e27c1da440dc20858ff55fe42211a1930c2d51bbdce09f430c7f1 0.0s + => => naming to docker.io/library/go-api:latest 0.0s + => => unpacking to docker.io/library/go-api:latest 0.0s + => [api] resolving provenance for metadata file 0.0s +service(s) ["api"] successfully built +``` + +### Testing the application + +Now that you have your application running, head over to the Grafana dashboard to visualize the metrics you are registering. Open your browser and navigate to `http://localhost:3000`. You will be greeted with the Grafana login page. The login credentials are the ones provided in Compose file. + +Once you are logged in, you can create a new dashboard. While creating dashboard you will notice that is default data source is `Prometheus`. This is because you have already configured the data source in the `grafana.yml` file. + +![The optional settings screen with the options specified.](../images/grafana-dash.png) + +You can use different panels to visualize the metrics. This guide doesn't go into details of Grafana. You can refer to the [Grafana documentation](https://grafana.com/docs/grafana/latest/) for more information. There is a Bar Gauge panel to visualize the total number of requests from different endpoints. You used the `api_http_request_total` and `api_http_request_error_total` metrics to get the data. + +![The optional settings screen with the options specified.](../images/grafana-panel.png) + +You created this panel to visualize the total number of requests from different endpoints to compare the successful and failed requests. For all the good requests, the bar will be green, and for all the failed requests, the bar will be red. Plus it will also show the from which endpoint the request is coming, either it's a successful request or a failed request. If you want to use this panel, you can import the `dashboard.json` file from the repository you cloned. + +### Summary + +You've come to the end of this guide. You learned how to develop the Golang application with Docker. You also saw how to use Docker Compose Watch to rebuild the image whenever you make changes to the code. Lastly, you tested the application and visualized the metrics in Grafana using Prometheus as the data source. \ No newline at end of file diff --git a/content/guides/go-prometheus-monitoring/application.md b/content/guides/go-prometheus-monitoring/application.md deleted file mode 100644 index 9845b9e127e1..000000000000 --- a/content/guides/go-prometheus-monitoring/application.md +++ /dev/null @@ -1,250 +0,0 @@ ---- -title: Building the application -linkTitle: Understand the application -weight: 10 # -keywords: go, golang, prometheus, grafana, containerize, monitor -description: Learn how to create a Golang server to register metrics with Prometheus. ---- - -## Prerequisites - -* You have a [Git client](https://git-scm.com/downloads). The examples in this section use a command-line based Git client, but you can use any client. - -You will be creating a Golang server with some endpoints to simulate a real-world application. Then you will expose metrics from the server using Prometheus. - -## Getting the sample application - -Clone the sample application to use with this guide. Open a terminal, change -directory to a directory that you want to work in, and run the following -command to clone the repository: - -```console -$ git clone https://github.com/dockersamples/go-prometheus-monitoring.git -``` - -Once you cloned you will see the following content structure inside `go-prometheus-monitoring` directory, - -```text -go-prometheus-monitoring -├── CONTRIBUTING.md -├── Docker -│ ├── grafana.yml -│ └── prometheus.yml -├── dashboard.json -├── Dockerfile -├── LICENSE -├── README.md -├── compose.yaml -├── go.mod -├── go.sum -└── main.go -``` - -- **main.go** - The entry point of the application. -- **go.mod and go.sum** - Go module files. -- **Dockerfile** - Dockerfile used to build the app. -- **Docker/** - Contains the Docker Compose configuration files for Grafana and Prometheus. -- **compose.yaml** - Compose file to launch everything (Golang app, Prometheus, and Grafana). -- **dashboard.json** - Grafana dashboard configuration file. -- **Dockerfile** - Dockerfile used to build the Golang app. -- **compose.yaml** - Docker Compose file to launch everything (Golang app, Prometheus, and Grafana). -- Other files are for licensing and documentation purposes. - -## Understanding the application - -The following is the complete logic of the application you will find in `main.go`. - -```go -package main - -import ( - "strconv" - - "github.com/gin-gonic/gin" - "github.com/prometheus/client_golang/prometheus" - "github.com/prometheus/client_golang/prometheus/promhttp" -) - -// Define metrics -var ( - HttpRequestTotal = prometheus.NewCounterVec(prometheus.CounterOpts{ - Name: "api_http_request_total", - Help: "Total number of requests processed by the API", - }, []string{"path", "status"}) - - HttpRequestErrorTotal = prometheus.NewCounterVec(prometheus.CounterOpts{ - Name: "api_http_request_error_total", - Help: "Total number of errors returned by the API", - }, []string{"path", "status"}) -) - -// Custom registry (without default Go metrics) -var customRegistry = prometheus.NewRegistry() - -// Register metrics with custom registry -func init() { - customRegistry.MustRegister(HttpRequestTotal, HttpRequestErrorTotal) -} - -func main() { - router := gin.Default() - - // Register /metrics before middleware - router.GET("/metrics", PrometheusHandler()) - - router.Use(RequestMetricsMiddleware()) - router.GET("/health", func(c *gin.Context) { - c.JSON(200, gin.H{ - "message": "Up and running!", - }) - }) - router.GET("/v1/users", func(c *gin.Context) { - c.JSON(200, gin.H{ - "message": "Hello from /v1/users", - }) - }) - - router.Run(":8000") -} - -// Custom metrics handler with custom registry -func PrometheusHandler() gin.HandlerFunc { - h := promhttp.HandlerFor(customRegistry, promhttp.HandlerOpts{}) - return func(c *gin.Context) { - h.ServeHTTP(c.Writer, c.Request) - } -} - -// Middleware to record incoming requests metrics -func RequestMetricsMiddleware() gin.HandlerFunc { - return func(c *gin.Context) { - path := c.Request.URL.Path - c.Next() - status := c.Writer.Status() - if status < 400 { - HttpRequestTotal.WithLabelValues(path, strconv.Itoa(status)).Inc() - } else { - HttpRequestErrorTotal.WithLabelValues(path, strconv.Itoa(status)).Inc() - } - } -} -``` - -In this part of the code, you have imported the required packages `gin`, `prometheus`, and `promhttp`. Then you have defined a couple of variables, `HttpRequestTotal` and `HttpRequestErrorTotal` are Prometheus counter metrics, and `customRegistry` is a custom registry that will be used to register these metrics. The name of the metric is a string that you can use to identify the metric. The help string is a string that will be shown when you query the `/metrics` endpoint to understand the metric. The reason you are using the custom registry is so avoid the default Go metrics that are registered by default by the Prometheus client. Then using the `init` function you are registering the metrics with the custom registry. - -```go -import ( - "strconv" - - "github.com/gin-gonic/gin" - "github.com/prometheus/client_golang/prometheus" - "github.com/prometheus/client_golang/prometheus/promhttp" -) - -// Define metrics -var ( - HttpRequestTotal = prometheus.NewCounterVec(prometheus.CounterOpts{ - Name: "api_http_request_total", - Help: "Total number of requests processed by the API", - }, []string{"path", "status"}) - - HttpRequestErrorTotal = prometheus.NewCounterVec(prometheus.CounterOpts{ - Name: "api_http_request_error_total", - Help: "Total number of errors returned by the API", - }, []string{"path", "status"}) -) - -// Custom registry (without default Go metrics) -var customRegistry = prometheus.NewRegistry() - -// Register metrics with custom registry -func init() { - customRegistry.MustRegister(HttpRequestTotal, HttpRequestErrorTotal) -} -``` - -In the `main` function, you have created a new instance of the `gin` framework and created three routes. You can see the health endpoint that is on path `/health` that will return a JSON with `{"message": "Up and running!"}` and the `/v1/users` endpoint that will return a JSON with `{"message": "Hello from /v1/users"}`. The third route is for the `/metrics` endpoint that will return the metrics in the Prometheus format. Then you have `RequestMetricsMiddleware` middleware, it will be called for every request made to the API. It will record the incoming requests metrics like status codes and paths. Finally, you are running the gin application on port 8000. - -```golang -func main() { - router := gin.Default() - - // Register /metrics before middleware - router.GET("/metrics", PrometheusHandler()) - - router.Use(RequestMetricsMiddleware()) - router.GET("/health", func(c *gin.Context) { - c.JSON(200, gin.H{ - "message": "Up and running!", - }) - }) - router.GET("/v1/users", func(c *gin.Context) { - c.JSON(200, gin.H{ - "message": "Hello from /v1/users", - }) - }) - - router.Run(":8000") -} -``` - -Now comes the middleware function `RequestMetricsMiddleware`. This function is called for every request made to the API. It increments the `HttpRequestTotal` counter (different counter for different paths and status codes) if the status code is less than or equal to 400. If the status code is greater than 400, it increments the `HttpRequestErrorTotal` counter (different counter for different paths and status codes). The `PrometheusHandler` function is the custom handler that will be called for the `/metrics` endpoint. It will return the metrics in the Prometheus format. - -```golang -// Custom metrics handler with custom registry -func PrometheusHandler() gin.HandlerFunc { - h := promhttp.HandlerFor(customRegistry, promhttp.HandlerOpts{}) - return func(c *gin.Context) { - h.ServeHTTP(c.Writer, c.Request) - } -} - -// Middleware to record incoming requests metrics -func RequestMetricsMiddleware() gin.HandlerFunc { - return func(c *gin.Context) { - path := c.Request.URL.Path - c.Next() - status := c.Writer.Status() - if status < 400 { - HttpRequestTotal.WithLabelValues(path, strconv.Itoa(status)).Inc() - } else { - HttpRequestErrorTotal.WithLabelValues(path, strconv.Itoa(status)).Inc() - } - } -} -``` - -That's it, this was the complete gist of the application. Now it's time to run and test if the app is registering metrics correctly. - -## Running the application - -Make sure you are still inside `go-prometheus-monitoring` directory in the terminal, and run the following command. Install the dependencies by running `go mod tidy` and then build and run the application by running `go run main.go`. Then visit `http://localhost:8000/health` or `http://localhost:8000/v1/users`. You should see the output `{"message": "Up and running!"}` or `{"message": "Hello from /v1/users"}`. If you are able to see this then your app is successfully up and running. - - -Now, check your application's metrics by accessing the `/metrics` endpoint. -Open `http://localhost:8000/metrics` in your browser. You should see similar output to the following. - -```sh -# HELP api_http_request_error_total Total number of errors returned by the API -# TYPE api_http_request_error_total counter -api_http_request_error_total{path="/",status="404"} 1 -api_http_request_error_total{path="//v1/users",status="404"} 1 -api_http_request_error_total{path="/favicon.ico",status="404"} 1 -# HELP api_http_request_total Total number of requests processed by the API -# TYPE api_http_request_total counter -api_http_request_total{path="/health",status="200"} 2 -api_http_request_total{path="/v1/users",status="200"} 1 -``` - -In the terminal, press `ctrl` + `c` to stop the application. - -> [!Note] -> If you don't want to run the application locally, and want to run it in a Docker container, skip to next page where you create a Dockerfile and containerize the application. - -## Summary - -In this section, you learned how to create a Golang app to register metrics with Prometheus. By implementing middleware functions, you were able to increment the counters based on the request path and status codes. - -## Next steps - -In the next section, you'll learn how to containerize your application. diff --git a/content/guides/go-prometheus-monitoring/compose.md b/content/guides/go-prometheus-monitoring/compose.md deleted file mode 100644 index c4dcbbab6122..000000000000 --- a/content/guides/go-prometheus-monitoring/compose.md +++ /dev/null @@ -1,166 +0,0 @@ ---- -title: Connecting services with Docker Compose -linkTitle: Connect with Compose -weight: 30 # -keywords: go, golang, prometheus, grafana, containerize, monitor -description: Learn how to connect services with Docker Compose to monitor a Golang application with Prometheus and Grafana. ---- - -Now that you have containerized the Golang application, you will use Docker Compose to connect your services together. You will connect the Golang application, Prometheus, and Grafana services together to monitor the Golang application with Prometheus and Grafana. - -## Creating a Docker Compose file - -Create a new file named `compose.yml` in the root directory of your Golang application. The Docker Compose file contains instructions to run multiple services and connect them together. - -Here is a Docker Compose file for a project that uses Golang, Prometheus, and Grafana. You will also find this file in the `go-prometheus-monitoring` directory. - -```yaml -services: - api: - container_name: go-api - build: - context: . - dockerfile: Dockerfile - image: go-api:latest - ports: - - 8000:8000 - networks: - - go-network - healthcheck: - test: ["CMD", "curl", "-f", "http://localhost:8000/health"] - interval: 30s - timeout: 10s - retries: 5 - develop: - watch: - - path: . - action: rebuild - - prometheus: - container_name: prometheus - image: prom/prometheus:v2.55.0 - volumes: - - ./Docker/prometheus.yml:/etc/prometheus/prometheus.yml - ports: - - 9090:9090 - networks: - - go-network - - grafana: - container_name: grafana - image: grafana/grafana:11.3.0 - volumes: - - ./Docker/grafana.yml:/etc/grafana/provisioning/datasources/datasource.yaml - - grafana-data:/var/lib/grafana - ports: - - 3000:3000 - networks: - - go-network - environment: - - GF_SECURITY_ADMIN_USER=admin - - GF_SECURITY_ADMIN_PASSWORD=password - -volumes: - grafana-data: - -networks: - go-network: - driver: bridge -``` - -## Understanding the Docker Compose file - -The Docker Compose file consists of three services: - -- **Golang application service**: This service builds the Golang application using the Dockerfile and runs it in a container. It exposes the application's port `8000` and connects to the `go-network` network. It also defines a health check to monitor the application's health. You have also used `healthcheck` to monitor the health of the application. The health check runs every 30 seconds and retries 5 times if the health check fails. The health check uses the `curl` command to check the `/health` endpoint of the application. Apart from the health check, you have also added a `develop` section to watch the changes in the application's source code and rebuild the application using the Docker Compose Watch feature. - -- **Prometheus service**: This service runs the Prometheus server in a container. It uses the official Prometheus image `prom/prometheus:v2.55.0`. It exposes the Prometheus server on port `9090` and connects to the `go-network` network. You have also mounted the `prometheus.yml` file from the `Docker` directory which is present in the root directory of your project. The `prometheus.yml` file contains the Prometheus configuration to scrape the metrics from the Golang application. This is how you connect the Prometheus server to the Golang application. - - ```yaml - global: - scrape_interval: 10s - evaluation_interval: 10s - - scrape_configs: - - job_name: myapp - static_configs: - - targets: ["api:8000"] - ``` - - In the `prometheus.yml` file, you have defined a job named `myapp` to scrape the metrics from the Golang application. The `targets` field specifies the target to scrape the metrics from. In this case, the target is the Golang application running on port `8000`. The `api` is the service name of the Golang application in the Docker Compose file. The Prometheus server will scrape the metrics from the Golang application every 10 seconds. - -- **Grafana service**: This service runs the Grafana server in a container. It uses the official Grafana image `grafana/grafana:11.3.0`. It exposes the Grafana server on port `3000` and connects to the `go-network` network. You have also mounted the `grafana.yml` file from the `Docker` directory which is present in the root directory of your project. The `grafana.yml` file contains the Grafana configuration to add the Prometheus data source. This is how you connect the Grafana server to the Prometheus server. In the environment variables, you have set the Grafana admin user and password, which will be used to log in to the Grafana dashboard. - - ```yaml - apiVersion: 1 - datasources: - - name: Prometheus (Main) - type: prometheus - url: http://prometheus:9090 - isDefault: true - ``` - - In the `grafana.yml` file, you have defined a Prometheus data source named `Prometheus (Main)`. The `type` field specifies the type of the data source, which is `prometheus`. The `url` field specifies the URL of the Prometheus server to fetch the metrics from. In this case, the URL is `http://prometheus:9090`. `prometheus` is the service name of the Prometheus server in the Docker Compose file. The `isDefault` field specifies whether the data source is the default data source in Grafana. - -Apart from the services, the Docker Compose file also defines a volume named `grafana-data` to persist the Grafana data and a network named `go-network` to connect the services together. You have created a custom network `go-network` to connect the services together. The `driver: bridge` field specifies the network driver to use for the network. - -## Building and running the services - -Now that you have the Docker Compose file, you can build the services and run them together using Docker Compose. - -To build and run the services, run the following command in the terminal: - -```console -$ docker compose up -``` - -The `docker compose up` command builds the services defined in the Docker Compose file and runs them together. You will see the similar output in the terminal: - -```console - ✔ Network go-prometheus-monitoring_go-network Created 0.0s - ✔ Container grafana Created 0.3s - ✔ Container go-api Created 0.2s - ✔ Container prometheus Created 0.3s -Attaching to go-api, grafana, prometheus -go-api | [GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached. -go-api | -go-api | [GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production. -go-api | - using env: export GIN_MODE=release -go-api | - using code: gin.SetMode(gin.ReleaseMode) -go-api | -go-api | [GIN-debug] GET /metrics --> main.PrometheusHandler.func1 (3 handlers) -go-api | [GIN-debug] GET /health --> main.main.func1 (4 handlers) -go-api | [GIN-debug] GET /v1/users --> main.main.func2 (4 handlers) -go-api | [GIN-debug] [WARNING] You trusted all proxies, this is NOT safe. We recommend you to set a value. -go-api | Please check https://pkg.go.dev/github.com/gin-gonic/gin#readme-don-t-trust-all-proxies for details. -go-api | [GIN-debug] Listening and serving HTTP on :8000 -prometheus | ts=2025-03-15T05:57:06.676Z caller=main.go:627 level=info msg="No time or size retention was set so using the default time retention" duration=15d -prometheus | ts=2025-03-15T05:57:06.678Z caller=main.go:671 level=info msg="Starting Prometheus Server" mode=server version="(version=2.55.0, branch=HEAD, revision=91d80252c3e528728b0f88d254dd720f6be07cb8)" -grafana | logger=settings t=2025-03-15T05:57:06.865335506Z level=info msg="Config overridden from command line" arg="default.log.mode=console" -grafana | logger=settings t=2025-03-15T05:57:06.865337131Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" -grafana | logger=ngalert.state.manager t=2025-03-15T05:57:07.088956839Z level=info msg="State -. -. -grafana | logger=plugin.angulardetectorsprovider.dynamic t=2025-03-15T05:57:07.530317298Z level=info msg="Patterns update finished" duration=440.489125ms -``` - -The services will start running, and you can access the Golang application at `http://localhost:8000`, Prometheus at `http://localhost:9090/health`, and Grafana at `http://localhost:3000`. You can also check the running containers using the `docker ps` command. - -```console -$ docker ps -``` - -## Summary - -In this section, you learned how to connect services together using Docker Compose. You created a Docker Compose file to run multiple services together and connect them using networks. You also learned how to build and run the services using Docker Compose. - -Related information: - - - [Docker Compose overview](/manuals/compose/_index.md) - - [Compose file reference](/reference/compose-file/_index.md) - -Next, you will learn how to develop the Golang application with Docker Compose and monitor it with Prometheus and Grafana. - -## Next steps - -In the next section, you will learn how to develop the Golang application with Docker. You will also learn how to use Docker Compose Watch to rebuild the image whenever you make changes to the code. Lastly, you will test the application and visualize the metrics in Grafana using Prometheus as the data source. diff --git a/content/guides/go-prometheus-monitoring/containerize.md b/content/guides/go-prometheus-monitoring/containerize.md deleted file mode 100644 index a628c380618f..000000000000 --- a/content/guides/go-prometheus-monitoring/containerize.md +++ /dev/null @@ -1,103 +0,0 @@ ---- -title: Containerize a Golang application -linkTitle: Containerize your app -weight: 20 -keywords: go, golang, containerize, initialize -description: Learn how to containerize a Golang application. ---- - -Containerization helps you bundle the application and its dependencies into a single package called a container. This package can run on any platform without worrying about the environment. In this section, you will learn how to containerize a Golang application using Docker. - -To containerize a Golang application, you first need to create a Dockerfile. The Dockerfile contains instructions to build and run the application in a container. Also, when creating a Dockerfile, you can follow different sets of best practices to optimize the image size and make it more secure. - -## Creating a Dockerfile - -Create a new file named `Dockerfile` in the root directory of your Golang application. The Dockerfile contains instructions to build and run the application in a container. - -The following is a Dockerfile for a Golang application. You will also find this file in the `go-prometheus-monitoring` directory. - -```dockerfile -# Use the official Golang image as the base -FROM golang:1.24-alpine AS builder - -# Set environment variables -ENV CGO_ENABLED=0 \ - GOOS=linux \ - GOARCH=amd64 - -# Set working directory inside the container -WORKDIR /build - -# Copy go.mod and go.sum files for dependency installation -COPY go.mod go.sum ./ - -# Download dependencies -RUN go mod download - -# Copy the entire application source -COPY . . - -# Build the Go binary -RUN go build -o /app . - -# Final lightweight stage -FROM alpine:3.21 AS final - -# Copy the compiled binary from the builder stage -COPY --from=builder /app /bin/app - -# Expose the application's port -EXPOSE 8000 - -# Run the application -CMD ["bin/app"] -``` - -## Understanding the Dockerfile - -The Dockerfile consists of two stages: - -1. **Build stage**: This stage uses the official Golang image as the base and sets the necessary environment variables. It also sets the working directory inside the container, copies the `go.mod` and `go.sum` files for dependency installation, downloads the dependencies, copies the entire application source, and builds the Go binary. - - You use the `golang:1.24-alpine` image as the base image for the build stage. The `CGO_ENABLED=0` environment variable disables CGO, which is useful for building static binaries. You also set the `GOOS` and `GOARCH` environment variables to `linux` and `amd64`, respectively, to build the binary for the Linux platform. - -2. **Final stage**: This stage uses the official Alpine image as the base and copies the compiled binary from the build stage. It also exposes the application's port and runs the application. - - You use the `alpine:3.21` image as the base image for the final stage. You copy the compiled binary from the build stage to the final image. You expose the application's port using the `EXPOSE` instruction and run the application using the `CMD` instruction. - - Apart from the multi-stage build, the Dockerfile also follows best practices such as using the official images, setting the working directory, and copying only the necessary files to the final image. You can further optimize the Dockerfile by other best practices. - -## Build the Docker image and run the application - -One you have the Dockerfile, you can build the Docker image and run the application in a container. - -To build the Docker image, run the following command in the terminal: - -```console -$ docker build -t go-api:latest . -``` - -After building the image, you can run the application in a container using the following command: - -```console -$ docker run -p 8000:8000 go-api:latest -``` - -The application will start running inside the container, and you can access it at `http://localhost:8000`. You can also check the running containers using the `docker ps` command. - -```console -$ docker ps -``` - -## Summary - -In this section, you learned how to containerize a Golang application using a Dockerfile. You created a multi-stage Dockerfile to build and run the application in a container. You also learned about best practices to optimize the Docker image size and make it more secure. - -Related information: - - - [Dockerfile reference](/reference/dockerfile.md) - - [.dockerignore file](/reference/dockerfile.md#dockerignore-file) - -## Next steps - -In the next section, you will learn how to use Docker Compose to connect and run multiple services together to monitor a Golang application with Prometheus and Grafana. diff --git a/content/guides/go-prometheus-monitoring/develop.md b/content/guides/go-prometheus-monitoring/develop.md deleted file mode 100644 index 7cf147604f5d..000000000000 --- a/content/guides/go-prometheus-monitoring/develop.md +++ /dev/null @@ -1,84 +0,0 @@ ---- -title: Developing your application -linkTitle: Develop your app -weight: 40 -keywords: go, golang, containerize, initialize -description: Learn how to develop the Golang application with Docker. ---- - -In the last section, you saw how using Docker Compose, you can connect your services together. In this section, you will learn how to develop the Golang application with Docker. You will also see how to use Docker Compose Watch to rebuild the image whenever you make changes to the code. Lastly, you will test the application and visualize the metrics in Grafana using Prometheus as the data source. - -## Developing the application - -Now, if you make any changes to your Golang application locally, it needs to reflect in the container, right? To do that, one approach is use the `--build` flag in Docker Compose after making changes in the code. This will rebuild all the services which have the `build` instruction in the `compose.yml` file, in your case, the `api` service (Golang application). - -```console -docker compose up --build -``` - -But, this is not the best approach. This is not efficient. Every time you make a change in the code, you need to rebuild manually. This is not is not a good flow for development. - -The better approach is to use Docker Compose Watch. In the `compose.yml` file, under the service `api`, you have added the `develop` section. So, it's more like a hot reloading. Whenever you make changes to code (defined in `path`), it will rebuild the image (or restart depending on the action). This is how you can use it: - -```yaml {hl_lines="17-20",linenos=true} -services: - api: - container_name: go-api - build: - context: . - dockerfile: Dockerfile - image: go-api:latest - ports: - - 8000:8000 - networks: - - go-network - healthcheck: - test: ["CMD", "curl", "-f", "http://localhost:8080/health"] - interval: 30s - timeout: 10s - retries: 5 - develop: - watch: - - path: . - action: rebuild -``` - -Once you have added the `develop` section in the `compose.yml` file, you can use the following command to start the development server: - -```console -$ docker compose watch -``` - -Now, if you modify your `main.go` or any other file in the project, the `api` service will be rebuilt automatically. You will see the following output in the terminal: - -```bash -Rebuilding service(s) ["api"] after changes were detected... -[+] Building 8.1s (15/15) FINISHED docker:desktop-linux - => [api internal] load build definition from Dockerfile 0.0s - => => transferring dockerfile: 704B 0.0s - => [api internal] load metadata for docker.io/library/alpine:3.17 1.1s - . - => => exporting manifest list sha256:89ebc86fd51e27c1da440dc20858ff55fe42211a1930c2d51bbdce09f430c7f1 0.0s - => => naming to docker.io/library/go-api:latest 0.0s - => => unpacking to docker.io/library/go-api:latest 0.0s - => [api] resolving provenance for metadata file 0.0s -service(s) ["api"] successfully built -``` - -## Testing the application - -Now that you have your application running, head over to the Grafana dashboard to visualize the metrics you are registering. Open your browser and navigate to `http://localhost:3000`. You will be greeted with the Grafana login page. The login credentials are the ones provided in Compose file. - -Once you are logged in, you can create a new dashboard. While creating dashboard you will notice that is default data source is `Prometheus`. This is because you have already configured the data source in the `grafana.yml` file. - -![The optional settings screen with the options specified.](../images/grafana-dash.png) - -You can use different panels to visualize the metrics. This guide doesn't go into details of Grafana. You can refer to the [Grafana documentation](https://grafana.com/docs/grafana/latest/) for more information. There is a Bar Gauge panel to visualize the total number of requests from different endpoints. You used the `api_http_request_total` and `api_http_request_error_total` metrics to get the data. - -![The optional settings screen with the options specified.](../images/grafana-panel.png) - -You created this panel to visualize the total number of requests from different endpoints to compare the successful and failed requests. For all the good requests, the bar will be green, and for all the failed requests, the bar will be red. Plus it will also show the from which endpoint the request is coming, either it's a successful request or a failed request. If you want to use this panel, you can import the `dashboard.json` file from the repository you cloned. - -## Summary - -You've come to the end of this guide. You learned how to develop the Golang application with Docker. You also saw how to use Docker Compose Watch to rebuild the image whenever you make changes to the code. Lastly, you tested the application and visualized the metrics in Grafana using Prometheus as the data source. \ No newline at end of file diff --git a/content/guides/golang/_index.md b/content/guides/golang/_index.md index 7d63660706a7..cc3ea119d17d 100644 --- a/content/guides/golang/_index.md +++ b/content/guides/golang/_index.md @@ -5,16 +5,31 @@ description: Containerize Go apps using Docker keywords: docker, getting started, go, golang, language, dockerfile summary: | This guide teaches you how to containerize Go applications using Docker. -toc_min: 1 -toc_max: 2 aliases: - /language/golang/ - /guides/language/golang/ -languages: [go] + - /get-started/golang/build-images/ + - /language/golang/build-images/ + - /get-started/golang/run-containers/ + - /language/golang/run-containers/ + - /get-started/golang/develop/ + - /language/golang/develop/ + - /get-started/golang/run-tests/ + - /language/golang/run-tests/ + - /language/golang/configure-ci-cd/ + - /language/golang/deploy/ + - /guides/golang/build-images/ + - /guides/golang/configure-ci-cd/ + - /guides/golang/deploy/ + - /guides/golang/develop/ + - /guides/golang/run-containers/ + - /guides/golang/run-tests/ params: + tags: [cicd] time: 30 minutes --- + This guide will show you how to create, test, and deploy containerized Go applications using Docker. > **Acknowledgment** @@ -52,3 +67,1861 @@ Some familiarity with the command line is also expected. The aim of this guide is to provide enough examples and instructions for you to containerize your own Go application and deploy it into the Cloud. Start by building your first Go image. + +## Build your Go image + +### Overview + +In this section you're going to build a container image. The image includes +everything you need to run your application – the compiled application binary +file, the runtime, the libraries, and all other resources required by your +application. + +### Required software + +To complete this tutorial, you need the following: + +- Docker running locally. Follow the [instructions to download and install Docker](/manuals/desktop/_index.md). +- An IDE or a text editor to edit files. [Visual Studio Code](https://code.visualstudio.com/) is a free and popular choice but you can use anything you feel comfortable with. +- A Git client. This guide uses a command-line based `git` client, but you are free to use whatever works for you. +- A command-line terminal application. The examples shown in this module are from the Linux shell, but they should work in PowerShell, Windows Command Prompt, or OS X Terminal with minimal, if any, modifications. + +### Meet the example application + +The example application is a caricature of a microservice. It is purposefully trivial to keep focus on learning the basics of containerization for Go applications. + +The application offers two HTTP endpoints: + +- It responds with a string containing a heart symbol (`<3`) to requests to `/`. +- It responds with `{"Status" : "OK"}` JSON to a request to `/health`. + +It responds with HTTP error 404 to any other request. + +The application listens on a TCP port defined by the value of environment variable `PORT`. The default value is `8080`. + +The application is stateless. + +The complete source code for the application is on GitHub: [github.com/docker/docker-gs-ping](https://github.com/docker/docker-gs-ping). You are encouraged to fork it and experiment with it as much as you like. + +To continue, clone the application repository to your local machine: + +```console +$ git clone https://github.com/docker/docker-gs-ping +``` + +The application's `main.go` file is straightforward, if you are familiar with Go: + +```go +package main + +import ( + "net/http" + "os" + + "github.com/labstack/echo/v4" + "github.com/labstack/echo/v4/middleware" +) + +func main() { + + e := echo.New() + + e.Use(middleware.Logger()) + e.Use(middleware.Recover()) + + e.GET("/", func(c echo.Context) error { + return c.HTML(http.StatusOK, "Hello, Docker! <3") + }) + + e.GET("/health", func(c echo.Context) error { + return c.JSON(http.StatusOK, struct{ Status string }{Status: "OK"}) + }) + + httpPort := os.Getenv("PORT") + if httpPort == "" { + httpPort = "8080" + } + + e.Logger.Fatal(e.Start(":" + httpPort)) +} + +// Simple implementation of an integer minimum +// Adapted from: https://gobyexample.com/testing-and-benchmarking +func IntMin(a, b int) int { + if a < b { + return a + } + return b +} +``` + +### Create a Dockerfile for the application + +To build a container image with Docker, a `Dockerfile` with build instructions is required. + +Begin your `Dockerfile` with the (optional) parser directive line that instructs BuildKit to +interpret your file according to the grammar rules for the specified version of the syntax. + +You then tell Docker what base image you would like to use for your application: + +```dockerfile +# syntax=docker/dockerfile:1 + +FROM golang:1.19 +``` + +Docker images can be inherited from other images. Therefore, instead of creating +your own base image from scratch, you can use the official Go image that already +has all necessary tools and libraries to compile and run a Go application. + +> [!NOTE] +> +> If you are curious about creating your own base images, you can check out the following section of this guide: [creating base images](/manuals/build/building/base-images.md#create-a-base-image). +> Note, however, that this isn't necessary to continue with your task at hand. + +Now that you have defined the base image for your upcoming container image, you +can begin building on top of it. + +To make things easier when running the rest of your commands, create a directory +inside the image that you're building. This also instructs Docker to use this +directory as the default destination for all subsequent commands. This way you +don't have to type out full file paths in the `Dockerfile`, the relative paths +will be based on this directory. + +```dockerfile +WORKDIR /app +``` + +Usually the very first thing you do once you’ve downloaded a project written in +Go is to install the modules necessary to compile it. Note, that the base image +has the toolchain already, but your source code isn't in it yet. + +So before you can run `go mod download` inside your image, you need to get your +`go.mod` and `go.sum` files copied into it. Use the `COPY` command to do this. + +In its simplest form, the `COPY` command takes two parameters. The first +parameter tells Docker what files you want to copy into the image. The last +parameter tells Docker where you want that file to be copied to. + +Copy the `go.mod` and `go.sum` file into your project directory `/app` which, +owing to your use of `WORKDIR`, is the current directory (`./`) inside the +image. Unlike some modern shells that appear to be indifferent to the use of +trailing slash (`/`), and can figure out what the user meant (most of the time), +Docker's `COPY` command is quite sensitive in its interpretation of the trailing +slash. + +```dockerfile +COPY go.mod go.sum ./ +``` + +> [!NOTE] +> +> If you'd like to familiarize yourself with the trailing slash treatment by the +> `COPY` command, see [Dockerfile +> reference](/reference/dockerfile.md#copy). This trailing slash can +> cause issues in more ways than you can imagine. + +Now that you have the module files inside the Docker image that you are +building, you can use the `RUN` command to run the command `go mod download` +there as well. This works exactly the same as if you were running `go` locally +on your machine, but this time these Go modules will be installed into a +directory inside the image. + +```dockerfile +RUN go mod download +``` + +At this point, you have a Go toolchain version 1.19.x and all your Go +dependencies installed inside the image. + +The next thing you need to do is to copy your source code into the image. You’ll +use the `COPY` command just like you did with your module files before. + +```dockerfile +COPY *.go ./ +``` + +This `COPY` command uses a wildcard to copy all files with `.go` extension +located in the current directory on the host (the directory where the `Dockerfile` +is located) into the current directory inside the image. + +Now, to compile your application, use the familiar `RUN` command: + +```dockerfile +RUN CGO_ENABLED=0 GOOS=linux go build -o /docker-gs-ping +``` + +This should be familiar. The result of that command will be a static application +binary named `docker-gs-ping` and located in the root of the filesystem of the +image that you are building. You could have put the binary into any other place +you desire inside that image, the root directory has no special meaning in this +regard. It's convenient to use it to keep the file paths short for improved +readability. + +Now, all that is left to do is to tell Docker what command to run when your +image is used to start a container. + +You do this with the `CMD` command: + +```dockerfile +CMD ["/docker-gs-ping"] +``` + +Here's the complete `Dockerfile`: + +```dockerfile +# syntax=docker/dockerfile:1 + +FROM golang:1.19 + +# Set destination for COPY +WORKDIR /app + +# Download Go modules +COPY go.mod go.sum ./ +RUN go mod download + +# Copy the source code. Note the slash at the end, as explained in +# https://docs.docker.com/reference/dockerfile/#copy +COPY *.go ./ + +# Build +RUN CGO_ENABLED=0 GOOS=linux go build -o /docker-gs-ping + +# Optional: +# To bind to a TCP port, runtime parameters must be supplied to the docker command. +# But we can document in the Dockerfile what ports +# the application is going to listen on by default. +# https://docs.docker.com/reference/dockerfile/#expose +EXPOSE 8080 + +# Run +CMD ["/docker-gs-ping"] +``` + +The `Dockerfile` may also contain comments. They always begin with a `#` symbol, +and must be at the beginning of a line. Comments are there for your convenience +to allow documenting your `Dockerfile`. + +There is also a concept of Dockerfile directives, such as the `syntax` directive +you added. The directives must always be at the very top of the `Dockerfile`, so +when adding comments, make sure that the comments follow after any directives +that you may have used: + +```dockerfile +# syntax=docker/dockerfile:1 +# A sample microservice in Go packaged into a container image. + +FROM golang:1.19 + +# ... +``` + +### Build the image + +Now that you've created your `Dockerfile`, build an image from it. The `docker +build` command creates Docker images from the `Dockerfile` and a context. A +build context is the set of files located in the specified path or URL. The +Docker build process can access any of the files located in the context. + +The build command optionally takes a `--tag` flag. This flag is used to label +the image with a string value, which is easy for humans to read and recognize. +If you don't pass a `--tag`, Docker will use `latest` as the default value. + +Build your first Docker image. + +```console +$ docker build --tag docker-gs-ping . +``` + +The build process will print some diagnostic messages as it goes through the build steps. +The following is an example of what these messages may look like. + +```console +[+] Building 2.2s (15/15) FINISHED + => [internal] load build definition from Dockerfile 0.0s + => => transferring dockerfile: 701B 0.0s + => [internal] load .dockerignore 0.0s + => => transferring context: 2B 0.0s + => resolve image config for docker.io/docker/dockerfile:1 1.1s + => CACHED docker-image://docker.io/docker/dockerfile:1@sha256:39b85bbfa7536a5feceb7372a0817649ecb2724562a38360f4d6a7782a409b14 0.0s + => [internal] load build definition from Dockerfile 0.0s + => [internal] load .dockerignore 0.0s + => [internal] load metadata for docker.io/library/golang:1.19 0.7s + => [1/6] FROM docker.io/library/golang:1.19@sha256:5d947843dde82ba1df5ac1b2ebb70b203d106f0423bf5183df3dc96f6bc5a705 0.0s + => [internal] load build context 0.0s + => => transferring context: 6.08kB 0.0s + => CACHED [2/6] WORKDIR /app 0.0s + => CACHED [3/6] COPY go.mod go.sum ./ 0.0s + => CACHED [4/6] RUN go mod download 0.0s + => CACHED [5/6] COPY *.go ./ 0.0s + => CACHED [6/6] RUN CGO_ENABLED=0 GOOS=linux go build -o /docker-gs-ping 0.0s + => exporting to image 0.0s + => => exporting layers 0.0s + => => writing image sha256:ede8ff889a0d9bc33f7a8da0673763c887a258eb53837dd52445cdca7b7df7e3 0.0s + => => naming to docker.io/library/docker-gs-ping 0.0s +``` + +Your exact output will vary, but provided there aren't any errors, you should +see the word `FINISHED` in the first line of output. This means Docker has +successfully built your image named `docker-gs-ping`. + +### View local images + +To see the list of images you have on your local machine, you have two options. +One is to use the CLI and the other is to use [Docker +Desktop](/manuals/desktop/_index.md). Since you're working in the +terminal, take a look at listing images with the CLI. + +To list images, run the `docker image ls`command (or the `docker images` shorthand): + +```console +$ docker image ls + +REPOSITORY TAG IMAGE ID CREATED SIZE +docker-gs-ping latest 7f153fbcc0a8 2 minutes ago 1.11GB +... +``` + +Your exact output may vary, but you should see the `docker-gs-ping` image with +the `latest` tag. Because you didn't specify a custom tag when you built your +image, Docker assumed that the tag would be `latest`, which is a special value. + +### Tag images + +An image name is made up of slash-separated name components. Name components may +contain lowercase letters, digits, and separators. A separator is defined as a +period, one or two underscores, or one or more dashes. A name component may not +start or end with a separator. + +An image is made up of a manifest and a list of layers. In simple terms, a tag +points to a combination of these artifacts. You can have multiple tags for the +image and, in fact, most images have multiple tags. Create a second tag +for the image you built and take a look at its layers. + +Use the `docker image tag` (or `docker tag` shorthand) command to create a new +tag for your image. This command takes two arguments; the first argument is the +source image, and the second is the new tag to create. The following command +creates a new `docker-gs-ping:v1.0` tag for the `docker-gs-ping:latest` you +built: + +```console +$ docker image tag docker-gs-ping:latest docker-gs-ping:v1.0 +``` + +The Docker `tag` command creates a new tag for the image. It doesn't create a +new image. The tag points to the same image and is another way to reference +the image. + +Now run the `docker image ls` command again to see the updated list of local +images: + +```console +$ docker image ls + +REPOSITORY TAG IMAGE ID CREATED SIZE +docker-gs-ping latest 7f153fbcc0a8 6 minutes ago 1.11GB +docker-gs-ping v1.0 7f153fbcc0a8 6 minutes ago 1.11GB +... +``` + +You can see that you have two images that start with `docker-gs-ping`. You know +they're the same image because if you look at the `IMAGE ID` column, you can +see that the values are the same for the two images. This value is a unique +identifier Docker uses internally to identify the image. + +Remove the tag that you just created. To do this, you’ll use the +`docker image rm` command, or the shorthand `docker rmi` (which stands for +"remove image"): + +```console +$ docker image rm docker-gs-ping:v1.0 +Untagged: docker-gs-ping:v1.0 +``` + +Notice that the response from Docker tells you that the image hasn't been +removed but only untagged. + +Verify this by running the following command: + +```console +$ docker image ls +``` + +You will see that the tag `v1.0` is no longer in the list of images kept by your Docker instance. + +```text +REPOSITORY TAG IMAGE ID CREATED SIZE +docker-gs-ping latest 7f153fbcc0a8 7 minutes ago 1.11GB +... +``` + +The tag `v1.0` has been removed but you still have the `docker-gs-ping:latest` +tag available on your machine, so the image is there. + +### Multi-stage builds + +You may have noticed that your `docker-gs-ping` image weighs in at over a +gigabyte, which is a lot for a tiny compiled Go application. You may also be +wondering what happened to the full suite of Go tools, including the compiler, +after you had built your image. + +The answer is that the full toolchain is still there, in the container image. +Not only this is inconvenient because of the large file size, but it may also +present a security risk when the container is deployed. + +These two issues can be solved by using [multi-stage builds](/manuals/build/building/multi-stage.md). + +In a nutshell, a multi-stage build can carry over the artifacts from one build stage into another, +and every build stage can be instantiated from a different base image. + +Thus, in the following example, you are going to use a full-scale official Go +image to build your application. Then you'll copy the application binary into +another image whose base is very lean and doesn't include the Go toolchain or +other optional components. + +The `Dockerfile.multistage` in the sample application's repository has the +following content: + +```dockerfile +# syntax=docker/dockerfile:1 + +# Build the application from source +FROM golang:1.19 AS build-stage + +WORKDIR /app + +COPY go.mod go.sum ./ +RUN go mod download + +COPY *.go ./ + +RUN CGO_ENABLED=0 GOOS=linux go build -o /docker-gs-ping + +# Run the tests in the container +FROM build-stage AS run-test-stage +RUN go test -v ./... + +# Deploy the application binary into a lean image +FROM gcr.io/distroless/base-debian11 AS build-release-stage + +WORKDIR / + +COPY --from=build-stage /docker-gs-ping /docker-gs-ping + +EXPOSE 8080 + +USER nonroot:nonroot + +ENTRYPOINT ["/docker-gs-ping"] +``` + +Since you have two Dockerfiles now, you have to tell Docker what Dockerfile +you'd like to use to build the image. Tag the new image with `multistage`. This +tag (like any other, apart from `latest`) has no special meaning for Docker, +it's something you chose. + +```console +$ docker build -t docker-gs-ping:multistage -f Dockerfile.multistage . +``` + +Comparing the sizes of `docker-gs-ping:multistage` and `docker-gs-ping:latest` +you see a few orders-of-magnitude difference. + +```console +$ docker image ls +REPOSITORY TAG IMAGE ID CREATED SIZE +docker-gs-ping multistage e3fdde09f172 About a minute ago 28.1MB +docker-gs-ping latest 336a3f164d0f About an hour ago 1.11GB +``` + +This is so because the ["distroless"](https://github.com/GoogleContainerTools/distroless) +base image that you have used in the second stage of the build is very barebones and is designed for lean deployments of static binaries. + +There's much more to multi-stage builds, including the possibility of multi-architecture builds, +so feel free to check out [multi-stage builds](/manuals/build/building/multi-stage.md). This is, however, not essential for your progress here. + +### Next steps + +In this module, you met your example application and built and container image +for it. + +In the next module, you’ll take a look at how to run your image as a container. + +## Run your Go image as a container + +### Prerequisites + +Work through the steps to containerize a Go application in [Build your Go image](build-images.md). + +### Overview + +In the previous module you created a `Dockerfile` for your example application and then you created your Docker image using the command `docker build`. Now that you have the image, you can run that image and see if your application is running correctly. + +A container is a normal operating system process except that this process is isolated and has its own file system, its own networking, and its own isolated process tree separate from the host. + +To run an image inside of a container, you use the `docker run` command. It requires one parameter and that's the image name. Start your image and make sure it's running correctly. Run the following command in your terminal. + +```console +$ docker run docker-gs-ping +``` + +```text + ____ __ + / __/___/ / ___ + / _// __/ _ \/ _ \ +/___/\__/_//_/\___/ v4.10.2 +High performance, minimalist Go web framework +https://echo.labstack.com +____________________________________O/_______ + O\ +⇨ http server started on [::]:8080 +``` + +When you run this command, you’ll notice that you weren't returned to the command prompt. This is because your application is a REST server and will run in a loop waiting for incoming requests without returning control back to the OS until you stop the container. + +Make a GET request to the server using the curl command. + +```console +$ curl http://localhost:8080/ +curl: (7) Failed to connect to localhost port 8080: Connection refused +``` + +Your curl command failed because the connection to your server was refused. +Meaning that you weren't able to connect to localhost on port 8080. This is +expected because your container is running in isolation which includes +networking. Stop the container and restart with port 8080 published on your +local network. + +To stop the container, press ctrl-c. This will return you to the terminal prompt. + +To publish a port for your container, you’ll use the `--publish` flag (`-p` for short) on the `docker run` command. The format of the `--publish` command is `[host_port]:[container_port]`. So if you wanted to expose port `8080` inside the container to port `3000` outside the container, you would pass `3000:8080` to the `--publish` flag. + +Start the container and expose port `8080` to port `8080` on the host. + +```console +$ docker run --publish 8080:8080 docker-gs-ping +``` + +Now, rerun the curl command. + +```console +$ curl http://localhost:8080/ +Hello, Docker! <3 +``` + +Success! You were able to connect to the application running inside of your container on port 8080. Switch back to the terminal where your container is running and you should see the `GET` request logged to the console. + +Press `ctrl-c` to stop the container. + +### Run in detached mode + +This is great so far, but your sample application is a web server and you +shouldn't have to have your terminal connected to the container. Docker can run +your container in detached mode in the background. To do this, you can use the +`--detach` or `-d` for short. Docker will start your container the same as +before but this time will detach from the container and return you to the +terminal prompt. + +```console +$ docker run -d -p 8080:8080 docker-gs-ping +d75e61fcad1e0c0eca69a3f767be6ba28a66625ce4dc42201a8a323e8313c14e +``` + +Docker started your container in the background and printed the container ID on the terminal. + +Again, make sure that your container is running. Run the same `curl` command: + +```console +$ curl http://localhost:8080/ +Hello, Docker! <3 +``` + +### List containers + +Since you ran your container in the background, how do you know if your container is running or what other containers are running on your machine? Well, to see a list of containers running on your machine, run `docker ps`. This is similar to how the ps command is used to see a list of processes on a Linux machine. + +```console +$ docker ps + +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +d75e61fcad1e docker-gs-ping "/docker-gs-ping" 41 seconds ago Up 40 seconds 0.0.0.0:8080->8080/tcp inspiring_ishizaka +``` + +The `ps` command tells you a bunch of stuff about your running containers. You can see the container ID, the image running inside the container, the command that was used to start the container, when it was created, the status, ports that are exposed, and the names of the container. + +You are probably wondering where the name of your container is coming from. Since you didn’t provide a name for the container when you started it, Docker generated a random name. You'll fix this in a minute but first you need to stop the container. To stop the container, run the `docker stop` command, passing the container's name or ID. + +```console +$ docker stop inspiring_ishizaka +inspiring_ishizaka +``` + +Now rerun the `docker ps` command to see a list of running containers. + +```console +$ docker ps + +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +``` + +### Stop, start, and name containers + +Docker containers can be started, stopped and restarted. When you stop a container, it's not removed but the status is changed to stopped and the process inside of the container is stopped. When you ran the `docker ps` command, the default output is to only show running containers. If you pass the `--all` or `-a` for short, you will see all containers on your system, including stopped containers and running containers. + +```console +$ docker ps --all + +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +d75e61fcad1e docker-gs-ping "/docker-gs-ping" About a minute ago Exited (2) 23 seconds ago inspiring_ishizaka +f65dbbb9a548 docker-gs-ping "/docker-gs-ping" 3 minutes ago Exited (2) 2 minutes ago wizardly_joliot +aade1bf3d330 docker-gs-ping "/docker-gs-ping" 3 minutes ago Exited (2) 3 minutes ago magical_carson +52d5ce3c15f0 docker-gs-ping "/docker-gs-ping" 9 minutes ago Exited (2) 3 minutes ago gifted_mestorf +``` + +If you’ve been following along, you should see several containers listed. These are containers that you started and stopped but haven't removed yet. + +Restart the container that you have just stopped. Locate the name of the container and replace the name of the container in the following `restart` command: + +```console +$ docker restart inspiring_ishizaka +``` + +Now, list all the containers again using the `ps` command: + +```console +$ docker ps -a + +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +d75e61fcad1e docker-gs-ping "/docker-gs-ping" 2 minutes ago Up 5 seconds 0.0.0.0:8080->8080/tcp inspiring_ishizaka +f65dbbb9a548 docker-gs-ping "/docker-gs-ping" 4 minutes ago Exited (2) 2 minutes ago wizardly_joliot +aade1bf3d330 docker-gs-ping "/docker-gs-ping" 4 minutes ago Exited (2) 4 minutes ago magical_carson +52d5ce3c15f0 docker-gs-ping "/docker-gs-ping" 10 minutes ago Exited (2) 4 minutes ago gifted_mestorf +``` + +Notice that the container you just restarted has been started in detached mode and has port `8080` exposed. Also, note that the status of the container is `Up X seconds`. When you restart a container, it will be started with the same flags or commands that it was originally started with. + +Stop and remove all of your containers and take a look at fixing the random naming issue. + +Stop the container you just started. Find the name of your running container and replace the name in the following command with the name of the container on your system: + +```console +$ docker stop inspiring_ishizaka +inspiring_ishizaka +``` + +Now that all of your containers are stopped, remove them. When a container is removed, it's no longer running nor is it in the stopped state. Instead, the process inside the container is terminated and the metadata for the container is removed. + +To remove a container, run the `docker rm` command passing the container name. You can pass multiple container names to the command in one command. + +Again, make sure you replace the containers names in the following command with the container names from your system: + +```console +$ docker rm inspiring_ishizaka wizardly_joliot magical_carson gifted_mestorf + +inspiring_ishizaka +wizardly_joliot +magical_carson +gifted_mestorf +``` + +Run the `docker ps --all` command again to verify that all containers are gone. + +Now, address the pesky random name issue. Standard practice is to name your containers for the simple reason that it's easier to identify what's running in the container and what application or service it's associated with. Just like good naming conventions for variables in your code makes it simpler to read. So goes naming your containers. + +To name a container, you must pass the `--name` flag to the `run` command: + +```console +$ docker run -d -p 8080:8080 --name rest-server docker-gs-ping +3bbc6a3102ea368c8b966e1878a5ea9b1fc61187afaac1276c41db22e4b7f48f +``` + +```console +$ docker ps + +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +3bbc6a3102ea docker-gs-ping "/docker-gs-ping" 25 seconds ago Up 24 seconds 0.0.0.0:8080->8080/tcp rest-server +``` + +Now, you can easily identify your container based on the name. + +### Next steps + +In this module, you learned how to run containers and publish ports. You also learned to manage the lifecycle of containers. You then learned the importance of naming your containers so that they're more easily identifiable. In the next module, you’ll learn how to run a database in a container and connect it to your application. + +## Use containers for Go development + +### Prerequisites + +Work through the steps of the [run your image as a container](run-containers.md) module to learn how to manage the lifecycle of your containers. + +### Introduction + +In this module, you'll take a look at running a database engine in a container and connecting it to the extended version of the example application. You are going to see some options for keeping persistent data and for wiring up the containers to talk to one another. Finally, you'll learn how to use Docker Compose to manage such multi-container local development environments effectively. + +### Local database and containers + +The database engine you are going to use is called [CockroachDB](https://www.cockroachlabs.com/product/). It is a modern, Cloud-native, distributed SQL database. + +Instead of compiling CockroachDB from the source code or using the operating system's native package manager to install CockroachDB, you are going to use the [Docker image for CockroachDB](https://hub.docker.com/r/cockroachdb/cockroach) and run it in a container. + +CockroachDB is compatible with PostgreSQL to a significant extent, and shares many conventions with the latter, particularly the default names for the environment variables. So, if you are familiar with Postgres, don't be surprised if you see some familiar environment variable names. The Go modules that work with Postgres, such as [pgx](https://pkg.go.dev/github.com/jackc/pgx), [pq](https://pkg.go.dev/github.com/lib/pq), [GORM](https://gorm.io/index.html), and [upper/db](https://upper.io/v4/) also work with CockroachDB. + +For more information on the relation between Go and CockroachDB, refer to the [CockroachDB documentation](https://www.cockroachlabs.com/docs/v20.2/build-a-go-app-with-cockroachdb.html), although this isn't necessary to continue with the present guide. + +#### Storage + +The point of a database is to have a persistent store of data. [Volumes](/manuals/engine/storage/volumes.md) are the preferred mechanism for persisting data generated by and used by Docker containers. Thus, before you start CockroachDB, create the volume for it. + +To create a managed volume, run : + +```console +$ docker volume create roach +roach +``` + +You can view the list of all managed volumes in your Docker instance with the following command: + +```console +$ docker volume list +DRIVER VOLUME NAME +local roach +``` + +#### Networking + +The example application and the database engine are going to talk to one another over the network. There are different kinds of network configuration possible, and you're going to use what's called a user-defined bridge network. It is going to provide you with a DNS lookup service so that you can refer to your database engine container by its hostname. + +The following command creates a new bridge network named `mynet`: + +```console +$ docker network create -d bridge mynet +51344edd6430b5acd121822cacc99f8bc39be63dd125a3b3cd517b6485ab7709 +``` + +As it was the case with the managed volumes, there is a command to list all networks set up in your Docker instance: + +```console +$ docker network list +NETWORK ID NAME DRIVER SCOPE +0ac2b1819fa4 bridge bridge local +51344edd6430 mynet bridge local +daed20bbecce host host local +6aee44f40a39 none null local +``` + +Your bridge network `mynet` has been created successfully. The other three networks, named `bridge`, `host`, and `none` are the default networks and they had been created by the Docker itself. While it's not relevant to this guide, you can learn more about Docker networking in the [networking overview](/manuals/engine/network/_index.md) section. + +#### Choose good names for volumes and networks + +As the saying goes, there are only two hard things in Computer Science: cache invalidation and naming things. And off-by-one errors. + +When choosing a name for a network or a managed volume, it's best to choose a name that's indicative of the intended purpose. This guide aims for brevity, so it used short, generic names. + +#### Start the database engine + +Now that the housekeeping chores are done, you can run CockroachDB in a container and attach it to the volume and network you had just created. When you run the following command, Docker will pull the image from Docker Hub and run it for you locally: + +```console +$ docker run -d \ + --name roach \ + --hostname db \ + --network mynet \ + -p 26257:26257 \ + -p 8080:8080 \ + -v roach:/cockroach/cockroach-data \ + cockroachdb/cockroach:latest-v25.4 start-single-node \ + --insecure + +# ... output omitted ... +``` + +Notice a clever use of the tag `latest-v25.4` to make sure that you're pulling the latest patch version of 25.4. The diversity of available tags depend on the image maintainer. Here, your intent was to have the latest patched version of CockroachDB while not straying too far away from the known working version as the time goes by. To see the tags available for the CockroachDB image, you can go to the [CockroachDB page on Docker Hub](https://hub.docker.com/r/cockroachdb/cockroach/tags). + +#### Configure the database engine + +Now that the database engine is live, there is some configuration to do before your application can begin using it. Fortunately, it's not a lot. You must: + +1. Create a blank database. +2. Register a new user account with the database engine. +3. Grant that new user access rights to the database. + +You can do that with the help of CockroachDB built-in SQL shell. To start the SQL shell in the same container where the database engine is running, type: + +```console +$ docker exec -it roach ./cockroach sql --insecure +``` + +1. In the SQL shell, create the database that the example application is going to use: + + ```sql + CREATE DATABASE mydb; + ``` + +2. Register a new SQL user account with the database engine. Use the username `totoro`. + + ```sql + CREATE USER totoro; + ``` + +3. Give the new user the necessary permissions: + + ```sql + GRANT ALL ON DATABASE mydb TO totoro; + ``` + +4. Type `quit` to exit the shell. + +The following is an example of interaction with the SQL shell. + +```console +$ sudo docker exec -it roach ./cockroach sql --insecure +# +# Welcome to the CockroachDB SQL shell. +# All statements must be terminated by a semicolon. +# To exit, type: \q. +# +# Server version: CockroachDB CCL v20.1.15 (x86_64-unknown-linux-gnu, built 2021/04/26 16:11:58, go1.13.9) (same version as client) +# Cluster ID: 7f43a490-ccd6-4c2a-9534-21f393ca80ce +# +# Enter \? for a brief introduction. +# +root@:26257/defaultdb> CREATE DATABASE mydb; +CREATE DATABASE + +Time: 22.985478ms + +root@:26257/defaultdb> CREATE USER totoro; +CREATE ROLE + +Time: 13.921659ms + +root@:26257/defaultdb> GRANT ALL ON DATABASE mydb TO totoro; +GRANT + +Time: 14.217559ms + +root@:26257/defaultdb> quit +oliver@hki:~$ +``` + +#### Meet the example application + +Now that you have started and configured the database engine, you can switch your attention to the application. + +The example application for this module is an extended version of `docker-gs-ping` application you've used in the previous modules. You have two options: + +- You can update your local copy of `docker-gs-ping` to match the new extended version presented in this chapter; or +- You can clone the [docker/docker-gs-ping-dev](https://github.com/docker/docker-gs-ping-dev) repository. This latter approach is recommended. + +To checkout the example application, run: + +```console +$ git clone https://github.com/docker/docker-gs-ping-dev.git +# ... output omitted ... +``` + +The application's `main.go` now includes database initialization code, as well as the code to implement a new business requirement: + +- An HTTP `POST` request to `/send` containing a `{ "value" : string }` JSON must save the value to the database. + +You also have an update for another business requirement. The requirement was: + +- The application responds with a text message containing a heart symbol ("`<3`") on requests to `/`. + +And now it's going to be: + +- The application responds with the string containing the count of messages stored in the database, enclosed in the parentheses. + + Example output: `Hello, Docker! (7)` + +The full source code listing of `main.go` follows. + +```go +package main + +import ( + "context" + "database/sql" + "fmt" + "log" + "net/http" + "os" + + "github.com/cenkalti/backoff/v4" + "github.com/cockroachdb/cockroach-go/v2/crdb" + "github.com/labstack/echo/v4" + "github.com/labstack/echo/v4/middleware" +) + +func main() { + + e := echo.New() + + e.Use(middleware.Logger()) + e.Use(middleware.Recover()) + + db, err := initStore() + if err != nil { + log.Fatalf("failed to initialize the store: %s", err) + } + defer db.Close() + + e.GET("/", func(c echo.Context) error { + return rootHandler(db, c) + }) + + e.GET("/ping", func(c echo.Context) error { + return c.JSON(http.StatusOK, struct{ Status string }{Status: "OK"}) + }) + + e.POST("/send", func(c echo.Context) error { + return sendHandler(db, c) + }) + + httpPort := os.Getenv("HTTP_PORT") + if httpPort == "" { + httpPort = "8080" + } + + e.Logger.Fatal(e.Start(":" + httpPort)) +} + +type Message struct { + Value string `json:"value"` +} + +func initStore() (*sql.DB, error) { + + pgConnString := fmt.Sprintf("host=%s port=%s dbname=%s user=%s password=%s sslmode=disable", + os.Getenv("PGHOST"), + os.Getenv("PGPORT"), + os.Getenv("PGDATABASE"), + os.Getenv("PGUSER"), + os.Getenv("PGPASSWORD"), + ) + + var ( + db *sql.DB + err error + ) + openDB := func() error { + db, err = sql.Open("postgres", pgConnString) + return err + } + + err = backoff.Retry(openDB, backoff.NewExponentialBackOff()) + if err != nil { + return nil, err + } + + if _, err := db.Exec( + "CREATE TABLE IF NOT EXISTS message (value TEXT PRIMARY KEY)"); err != nil { + return nil, err + } + + return db, nil +} + +func rootHandler(db *sql.DB, c echo.Context) error { + r, err := countRecords(db) + if err != nil { + return c.HTML(http.StatusInternalServerError, err.Error()) + } + return c.HTML(http.StatusOK, fmt.Sprintf("Hello, Docker! (%d)\n", r)) +} + +func sendHandler(db *sql.DB, c echo.Context) error { + + m := &Message{} + + if err := c.Bind(m); err != nil { + return c.JSON(http.StatusInternalServerError, err) + } + + err := crdb.ExecuteTx(context.Background(), db, nil, + func(tx *sql.Tx) error { + _, err := tx.Exec( + "INSERT INTO message (value) VALUES ($1) ON CONFLICT (value) DO UPDATE SET value = excluded.value", + m.Value, + ) + if err != nil { + return c.JSON(http.StatusInternalServerError, err) + } + return nil + }) + + if err != nil { + return c.JSON(http.StatusInternalServerError, err) + } + + return c.JSON(http.StatusOK, m) +} + +func countRecords(db *sql.DB) (int, error) { + + rows, err := db.Query("SELECT COUNT(*) FROM message") + if err != nil { + return 0, err + } + defer rows.Close() + + count := 0 + for rows.Next() { + if err := rows.Scan(&count); err != nil { + return 0, err + } + rows.Close() + } + + return count, nil +} +``` + +The repository also includes the `Dockerfile`, which is almost exactly the same as the multi-stage `Dockerfile` introduced in the previous modules. It uses the official Docker Go image to build the application and then builds the final image by placing the compiled binary into the much slimmer, distroless image. + +Regardless of whether you had updated the old example application, or checked out the new one, this new Docker image has to be built to reflect the changes to the application source code. + +#### Build the application + +You can build the image with the familiar `build` command: + +```console +$ docker build --tag docker-gs-ping-roach . +``` + +#### Run the application + +Now, run your container. This time you'll need to set some environment variables so that your application knows how to access the database. For now, you’ll do this right in the `docker run` command. Later you'll see a more convenient method with Docker Compose. + +> [!NOTE] +> +> Since you're running your CockroachDB cluster in insecure mode, the value for the password can be anything. +> +> In production, don't run in insecure mode. + +```console +$ docker run -it --rm -d \ + --network mynet \ + --name rest-server \ + -p 80:8080 \ + -e PGUSER=totoro \ + -e PGPASSWORD=myfriend \ + -e PGHOST=db \ + -e PGPORT=26257 \ + -e PGDATABASE=mydb \ + docker-gs-ping-roach +``` + +There are a few points to note about this command. + +- You map container port `8080` to host port `80` this time. Thus, for `GET` requests you can get away with literally `curl localhost`: + + ```console + $ curl localhost + Hello, Docker! (0) + ``` + + Or, if you prefer, a proper URL would work just as well: + + ```console + $ curl http://localhost/ + Hello, Docker! (0) + ``` + +- The total number of stored messages is `0` for now. This is fine, because you haven't posted anything to your application yet. +- You refer to the database container by its hostname, which is `db`. This is why you had `--hostname db` when you started the database container. + +- The actual password doesn't matter, but it must be set to something to avoid confusing the example application. +- The container you've just run is named `rest-server`. These names are useful for managing the container lifecycle: + + ```console + # Don't do this just yet, it's only an example: + $ docker container rm --force rest-server + ``` + +#### Test the application + +In the previous section, you've already tested querying your application with `GET` and it returned zero for the stored message counter. Now, post some messages to it: + +```console +$ curl --request POST \ + --url http://localhost/send \ + --header 'content-type: application/json' \ + --data '{"value": "Hello, Docker!"}' +``` + +The application responds with the contents of the message, which means it has been saved in the database: + +```json +{ "value": "Hello, Docker!" } +``` + +Send another message: + +```console +$ curl --request POST \ + --url http://localhost/send \ + --header 'content-type: application/json' \ + --data '{"value": "Hello, Oliver!"}' +``` + +And again, you get the value of the message back: + +```json +{ "value": "Hello, Oliver!" } +``` + +Run curl and see what the message counter says: + +```console +$ curl localhost +Hello, Docker! (2) +``` + +In this example, you sent two messages and the database kept them. Or has it? Stop and remove all your containers, but not the volumes, and try again. + +First, stop the containers: + +```console +$ docker container stop rest-server roach +rest-server +roach +``` + +Then, remove them: + +```console +$ docker container rm rest-server roach +rest-server +roach +``` + +Verify that they're gone: + +```console +$ docker container list --all +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +``` + +And start them again, database first: + +```console +$ docker run -d \ + --name roach \ + --hostname db \ + --network mynet \ + -p 26257:26257 \ + -p 8080:8080 \ + -v roach:/cockroach/cockroach-data \ + cockroachdb/cockroach:latest-v25.4 start-single-node \ + --insecure +``` + +And the service next: + +```console +$ docker run -it --rm -d \ + --network mynet \ + --name rest-server \ + -p 80:8080 \ + -e PGUSER=totoro \ + -e PGPASSWORD=myfriend \ + -e PGHOST=db \ + -e PGPORT=26257 \ + -e PGDATABASE=mydb \ + docker-gs-ping-roach +``` + +Lastly, query your service: + +```console +$ curl localhost +Hello, Docker! (2) +``` + +Great! The count of records from the database is correct although you haven't only stopped the containers, but you've also removed them before starting new instances. The difference is in the managed volume for CockroachDB, which you reused. The new CockroachDB container has read the database files from the disk, just as it normally would if it were running outside the container. + +#### Wind down everything + +Remember, that you're running CockroachDB in insecure mode. Now that you've built and tested your application, it's time to wind everything down before moving on. You can list the containers that you are running with the `list` command: + +```console +$ docker container list +``` + +Now that you know the container IDs, you can use `docker container stop` and `docker container rm`, as demonstrated in the previous modules. + +Stop the CockroachDB and `docker-gs-ping-roach` containers before moving on. + +### Better productivity with Docker Compose + +At this point, you might be wondering if there is a way to avoid having to deal with long lists of arguments to the `docker` command. The toy example you used in this series requires five environment variables to define the connection to the database. A real application might need many, many more. Then there is also a question of dependencies. Ideally, you want to make sure that the database is started before your application is run. And spinning up the database instance may require another Docker command with many options. But there is a better way to orchestrate these deployments for local development purposes. + +In this section, you'll create a Docker Compose file to start your `docker-gs-ping-roach` application and CockroachDB database engine with a single command. + +#### Configure Docker Compose + +In your application's directory, create a new text file named `compose.yaml` with the following content. + +```yaml +services: + docker-gs-ping-roach: + depends_on: + - roach + build: + context: . + container_name: rest-server + hostname: rest-server + networks: + - mynet + ports: + - 80:8080 + environment: + - PGUSER=${PGUSER:-totoro} + - PGPASSWORD=${PGPASSWORD:?database password not set} + - PGHOST=${PGHOST:-db} + - PGPORT=${PGPORT:-26257} + - PGDATABASE=${PGDATABASE:-mydb} + deploy: + restart_policy: + condition: on-failure + roach: + image: cockroachdb/cockroach:latest-v25.4 + container_name: roach + hostname: db + networks: + - mynet + ports: + - 26257:26257 + - 8080:8080 + volumes: + - roach:/cockroach/cockroach-data + command: start-single-node --insecure + +volumes: + roach: + +networks: + mynet: + driver: bridge +``` + +This Docker Compose configuration is super convenient as you don't have to type all the parameters to pass to the `docker run` command. You can declaratively do that in the Docker Compose file. The [Docker Compose documentation pages](/manuals/compose/_index.md) are quite extensive and include a full reference for the Docker Compose file format. + +#### The `.env` file + +Docker Compose will automatically read environment variables from a `.env` file if it's available. Since your Compose file requires `PGPASSWORD` to be set, add the following content to the `.env` file: + +```bash +PGPASSWORD=whatever +``` + +The exact value doesn't really matter for this example, because you run CockroachDB in insecure mode. Make sure you set the variable to some value to avoid getting an error. + +#### Merging Compose files + +The filename `compose.yaml` is the default filename which `docker compose` command recognizes if no `-f` flag is provided. This means you can have multiple Docker Compose files if your environment has such requirements. Furthermore, Docker Compose files are composable, so multiple files can be specified on the command line to merge parts of the configuration together. The following list shows a few examples of scenarios where such a feature would be useful: + +- Using a bind mount for the source code for local development but not when running the CI tests; +- Switching between using a pre-built image for the frontend for some API application vs creating a bind mount for source code; +- Adding additional services for integration testing; +- And many more... + +You aren't going to cover any of these advanced use cases here. + +#### Variable substitution in Docker Compose + +One of the really cool features of Docker Compose is [variable substitution](/reference/compose-file/interpolation.md). You can see some examples in the Compose file, `environment` section. By means of an example: + +- `PGUSER=${PGUSER:-totoro}` means that inside the container, the environment variable `PGUSER` shall be set to the same value as it has on the host machine where Docker Compose is run. If there is no environment variable with this name on the host machine, the variable inside the container gets the default value of `totoro`. +- `PGPASSWORD=${PGPASSWORD:?database password not set}` means that if the environment variable `PGPASSWORD` isn't set on the host, Docker Compose will display an error. This is OK, because you don't want to hard-code default values for the password. You set the password value in the `.env` file, which is local to your machine. It is always a good idea to add `.env` to `.gitignore` to prevent the secrets being checked into the version control. + +Other ways of dealing with undefined or empty values exist, as documented in the [variable substitution](/reference/compose-file/interpolation.md) section of the Docker documentation. + +#### Validating Docker Compose configuration + +Before you apply changes made to a Compose configuration file, there is an opportunity to validate the content of the configuration file with the following command: + +```console +$ docker compose config +``` + +When this command is run, Docker Compose reads the file `compose.yaml`, parses it into a data structure in memory, validates where possible, and prints back the reconstruction of that configuration file from its internal representation. If this isn't possible due to errors, Docker prints an error message instead. + +#### Build and run the application using Docker Compose + +Start your application and confirm that it's running. + +```console +$ docker compose up --build +``` + +You passed the `--build` flag so Docker will compile your image and then start it. + +> [!NOTE] +> +> Docker Compose is a useful tool, but it has its own quirks. For example, no rebuild is triggered on the update to the source code unless the `--build` flag is provided. It is a very common pitfall to edit one's source code, and forget to use the `--build` flag when running `docker compose up`. + +Since your set-up is now run by Docker Compose, it has assigned it a project name, so you get a new volume for your CockroachDB instance. This means that your application will fail to connect to the database, because the database doesn't exist in this new volume. The terminal displays an authentication error for the database: + +```text +# ... omitted output ... +rest-server | 2021/05/10 00:54:25 failed to initialise the store: pq: password authentication failed for user totoro +roach | * +roach | * INFO: Replication was disabled for this cluster. +roach | * When/if adding nodes in the future, update zone configurations to increase the replication factor. +roach | * +roach | CockroachDB node starting at 2021-05-10 00:54:26.398177 +0000 UTC (took 3.0s) +roach | build: CCL v20.1.15 @ 2021/04/26 16:11:58 (go1.13.9) +roach | webui: http://db:8080 +roach | sql: postgresql://root@db:26257?sslmode=disable +roach | RPC client flags: /cockroach/cockroach --host=db:26257 --insecure +roach | logs: /cockroach/cockroach-data/logs +roach | temp dir: /cockroach/cockroach-data/cockroach-temp349434348 +roach | external I/O path: /cockroach/cockroach-data/extern +roach | store[0]: path=/cockroach/cockroach-data +roach | storage engine: rocksdb +roach | status: initialized new cluster +roach | clusterID: b7b1cb93-558f-4058-b77e-8a4ddb329a88 +roach | nodeID: 1 +rest-server exited with code 0 +rest-server | 2021/05/10 00:54:25 failed to initialise the store: pq: password authentication failed for user totoro +rest-server | 2021/05/10 00:54:26 failed to initialise the store: pq: password authentication failed for user totoro +rest-server | 2021/05/10 00:54:29 failed to initialise the store: pq: password authentication failed for user totoro +rest-server | 2021/05/10 00:54:25 failed to initialise the store: pq: password authentication failed for user totoro +rest-server | 2021/05/10 00:54:26 failed to initialise the store: pq: password authentication failed for user totoro +rest-server | 2021/05/10 00:54:29 failed to initialise the store: pq: password authentication failed for user totoro +rest-server exited with code 1 +# ... omitted output ... +``` + +Because of the way you set up your deployment using `restart_policy`, the failing container is being restarted every 20 seconds. So, in order to fix the problem, you need to log in to the database engine and create the user. You've done it before in [Configure the database engine](#configure-the-database-engine). + +This isn't a big deal. All you have to do is to connect to CockroachDB instance and run the three SQL commands to create the database and the user, as described in [Configure the database engine](#configure-the-database-engine). + +So, log in to the database engine from another terminal: + +```console +$ docker exec -it roach ./cockroach sql --insecure +``` + +And run the same commands as before to create the database `mydb`, the user `totoro`, and to grant that user necessary permissions. Once you do that (and the example application container is automatically restarts), the `rest-service` stops failing and restarting and the console goes quiet. + +It would have been possible to connect the volume that you had previously used, but for the purposes of this example it's more trouble than it's worth and it also provided an opportunity to show how to introduce resilience into your deployment via the `restart_policy` Compose file feature. + +#### Testing the application + +Now, test your API endpoint. In the new terminal, run the following command: + +```console +$ curl http://localhost/ +``` + +You should receive the following response: + +```json +Hello, Docker! (0) +``` + +#### Shutting down + +To stop the containers started by Docker Compose, press `ctrl+c` in the terminal where you ran `docker compose up`. To remove those containers after they've been stopped, run `docker compose down`. + +#### Detached mode + +You can run containers started by the `docker compose` command in detached mode, just as you would with the `docker` command, by using the `-d` flag. + +To start the stack, defined by the Compose file in detached mode, run: + +```console +$ docker compose up --build -d +``` + +Then, you can use `docker compose stop` to stop the containers and `docker compose down` to remove them. + +### Further exploration + +You can run `docker compose` to see what other commands are available. + +### Wrap up + +There are some tangential, yet interesting points that were purposefully not covered in this chapter. For the more adventurous reader, this section offers some pointers for further study. + +#### Persistent storage + +A managed volume isn't the only way to provide your container with persistent storage. It is highly recommended to get acquainted with available storage options and their use cases, covered in [Manage data in Docker](/manuals/engine/storage/_index.md). + +#### CockroachDB clusters + +You ran a single instance of CockroachDB, which was enough for this example. But, it's possible to run a CockroachDB cluster, which is made of multiple instances of CockroachDB, each instance running in its own container. Since CockroachDB engine is distributed by design, it would have taken surprisingly little change to your procedure to run a cluster with multiple nodes. + +Such distributed set-up offers interesting possibilities, such as applying Chaos Engineering techniques to simulate parts of the cluster failing and evaluating your application's ability to cope with such failures. + +If you are interested in experimenting with CockroachDB clusters, check out: + +- [Start a CockroachDB Cluster in Docker](https://www.cockroachlabs.com/docs/v20.2/start-a-local-cluster-in-docker-mac.html) article; and +- Documentation for Docker Compose keywords [`deploy`](/reference/compose-file/legacy-versions.md) and [`replicas`](/reference/compose-file/legacy-versions.md). + +#### Other databases + +Since you didn't run a cluster of CockroachDB instances, you might be wondering whether you could have used a non-distributed database engine. The answer is 'yes', and if you were to pick a more traditional SQL database, such as [PostgreSQL](https://www.postgresql.org/), the process described in this chapter would have been very similar. + +### Next steps + +In this module, you set up a containerized development environment with your application and the database engine running in different containers. You also wrote a Docker Compose file which links the two containers together and provides for easy starting up and tearing down of the development environment. + +In the next module, you'll take a look at one possible approach to running functional tests in Docker. + +## Run your tests using Go test + +### Prerequisites + +Complete the [Build your Go image](build-images.md) section of this guide. + +### Overview + +Testing is an essential part of modern software development. Testing can mean a +lot of things to different development teams. There are unit tests, integration +tests and end-to-end testing. In this guide you take a look at running your unit +tests in Docker when building. + +For this section, use the `docker-gs-ping` project that you cloned in [Build +your Go image](build-images.md). + +### Run tests when building + +To run your tests when building, you need to add a test stage to the +`Dockerfile.multistage`. The `Dockerfile.multistage` in the sample application's +repository already has the following content: + +```dockerfile {hl_lines="15-17"} +# syntax=docker/dockerfile:1 + +# Build the application from source +FROM golang:1.19 AS build-stage + +WORKDIR /app + +COPY go.mod go.sum ./ +RUN go mod download + +COPY *.go ./ + +RUN CGO_ENABLED=0 GOOS=linux go build -o /docker-gs-ping + +# Run the tests in the container +FROM build-stage AS run-test-stage +RUN go test -v ./... + +# Deploy the application binary into a lean image +FROM gcr.io/distroless/base-debian11 AS build-release-stage + +WORKDIR / + +COPY --from=build-stage /docker-gs-ping /docker-gs-ping + +EXPOSE 8080 + +USER nonroot:nonroot + +ENTRYPOINT ["/docker-gs-ping"] +``` + +Run the following command to build an image using the `run-test-stage` stage as the target and view the test results. Include `--progress plain` to view the build output, `--no-cache` to ensure the tests always run, and `--target run-test-stage` to target the test stage. + +```console +$ docker build -f Dockerfile.multistage -t docker-gs-ping-test --progress plain --no-cache --target run-test-stage . +``` + +You should see output containing the following. + +```text +#13 [run-test-stage 1/1] RUN go test -v ./... +#13 4.915 === RUN TestIntMinBasic +#13 4.915 --- PASS: TestIntMinBasic (0.00s) +#13 4.915 === RUN TestIntMinTableDriven +#13 4.915 === RUN TestIntMinTableDriven/0,1 +#13 4.915 === RUN TestIntMinTableDriven/1,0 +#13 4.915 === RUN TestIntMinTableDriven/2,-2 +#13 4.915 === RUN TestIntMinTableDriven/0,-1 +#13 4.915 === RUN TestIntMinTableDriven/-1,0 +#13 4.915 --- PASS: TestIntMinTableDriven (0.00s) +#13 4.915 --- PASS: TestIntMinTableDriven/0,1 (0.00s) +#13 4.915 --- PASS: TestIntMinTableDriven/1,0 (0.00s) +#13 4.915 --- PASS: TestIntMinTableDriven/2,-2 (0.00s) +#13 4.915 --- PASS: TestIntMinTableDriven/0,-1 (0.00s) +#13 4.915 --- PASS: TestIntMinTableDriven/-1,0 (0.00s) +#13 4.915 PASS +``` + +### Next steps + +In this section, you learned how to run tests when building your image. Next, +you’ll learn how to set up a CI/CD pipeline using GitHub Actions. + +## Configure CI/CD for your Go application + +### Prerequisites + +Complete the previous sections of this guide, starting with [Build your Go image](build-images.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. + +### Overview + +In this section, you'll learn how to set up and use GitHub Actions to build and push your Docker image to Docker Hub. You will complete the following steps: + +1. Create a new repository on GitHub. +2. Define the GitHub Actions workflow. +3. Run the workflow. + +### Step one: Create the repository + +Create a GitHub repository, configure the Docker Hub credentials, and push your source code. + +1. [Create a new repository](https://github.com/new) on GitHub. + +2. Open the repository **Settings**, and go to **Secrets and variables** > + **Actions**. + +3. Create a new **Repository variable** named `DOCKER_USERNAME` and your Docker ID as a value. + +4. Create a new [Personal Access Token (PAT)](/manuals/security/access-tokens.md#create-an-access-token) for Docker Hub. You can name this token `docker-tutorial`. Make sure access permissions include Read and Write. + +5. Add the PAT as a **Repository secret** in your GitHub repository, with the name + `DOCKERHUB_TOKEN`. + +6. In your local repository on your machine, run the following command to change + the origin to the repository you just created. Make sure you change + `your-username` to your GitHub username and `your-repository` to the name of + the repository you created. + + ```console + $ git remote set-url origin https://github.com/your-username/your-repository.git + ``` + +7. Run the following commands to stage, commit, and push your local repository to GitHub. + + ```console + $ git add -A + $ git commit -m "my commit" + $ git push -u origin main + ``` + +### Step two: Set up the workflow + +Set up your GitHub Actions workflow for building, testing, and pushing the image +to Docker Hub. + +1. Go to your repository on GitHub and then select the **Actions** tab. + +2. Select **set up a workflow yourself**. + + This takes you to a page for creating a new GitHub actions workflow file in + your repository, under `.github/workflows/main.yml` by default. + +3. In the editor window, copy and paste the following YAML configuration and commit the changes. + + ```yaml + name: ci + + on: + push: + branches: + - main + + jobs: + build: + runs-on: ubuntu-latest + steps: + - name: Login to Docker Hub + uses: docker/login-action@{{% param "login_action_version" %}} + with: + username: ${{ vars.DOCKER_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} + + - name: Build and push + uses: docker/build-push-action@{{% param "build_push_action_version" %}} + with: + platforms: linux/amd64,linux/arm64 + push: true + tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest + ``` + + For more information about the YAML syntax for `docker/build-push-action`, + refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). + +### Step three: Run the workflow + +Save the workflow file and run the job. + +1. Select **Commit changes...** and push the changes to the `main` branch. + + After pushing the commit, the workflow starts automatically. + +2. Go to the **Actions** tab. It displays the workflow. + + Selecting the workflow shows you the breakdown of all the steps. + +3. When the workflow is complete, go to your + [repositories on Docker Hub](https://hub.docker.com/repositories). + + If you see the new repository in that list, it means the GitHub Actions + successfully pushed the image to Docker Hub. + +### Summary + +In this section, you learned how to set up a GitHub Actions workflow for your application. + +Related information: + +- [Introduction to GitHub Actions](/guides/gha.md) +- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) +- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) + +### Next steps + +Next, learn how you can locally test and debug your workloads on Kubernetes before deploying. + +## Test your Go deployment + +### Prerequisites + +- Complete all the previous sections of this guide, starting with [Build + your Go image](build-images.md). +- [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker + Desktop. + +### Overview + +In this section, you'll learn how to use Docker Desktop to deploy your +application to a fully-featured Kubernetes environment on your development +machine. This allows you to test and debug your workloads on Kubernetes locally +before deploying. + +### Create a Kubernetes YAML file + +In your project directory, create a file named +`docker-go-kubernetes.yaml`. Open the file in an IDE or text editor and add +the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker +username and the name of the repository that you created in [Configure CI/CD for +your Go application](configure-ci-cd.md). + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + labels: + service: server + name: server + namespace: default +spec: + replicas: 1 + selector: + matchLabels: + service: server + strategy: {} + template: + metadata: + labels: + service: server + spec: + initContainers: + - name: wait-for-db + image: busybox:1.28 + command: + [ + "sh", + "-c", + 'until nc -zv db 5432; do echo "waiting for db"; sleep 2; done;', + ] + containers: + - env: + - name: PGDATABASE + value: mydb + - name: PGPASSWORD + value: whatever + - name: PGHOST + value: db + - name: PGPORT + value: "5432" + - name: PGUSER + value: postgres + image: DOCKER_USERNAME/REPO_NAME + name: server + imagePullPolicy: Always + ports: + - containerPort: 8080 + hostPort: 8080 + protocol: TCP + resources: {} + restartPolicy: Always +status: {} +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + labels: + service: db + name: db + namespace: default +spec: + replicas: 1 + selector: + matchLabels: + service: db + strategy: + type: Recreate + template: + metadata: + labels: + service: db + spec: + containers: + - env: + - name: POSTGRES_DB + value: mydb + - name: POSTGRES_PASSWORD + value: whatever + - name: POSTGRES_USER + value: postgres + image: postgres:18 + name: db + ports: + - containerPort: 5432 + protocol: TCP + resources: {} + restartPolicy: Always +status: {} +--- +apiVersion: v1 +kind: Service +metadata: + labels: + service: server + name: server + namespace: default +spec: + type: NodePort + ports: + - name: "8080" + port: 8080 + targetPort: 8080 + nodePort: 30001 + selector: + service: server +status: + loadBalancer: {} +--- +apiVersion: v1 +kind: Service +metadata: + labels: + service: db + name: db + namespace: default +spec: + ports: + - name: "5432" + port: 5432 + targetPort: 5432 + selector: + service: db +status: + loadBalancer: {} +``` + +In this Kubernetes YAML file, there are four objects, separated by the `---`. In addition to a Service and Deployment for the database, the other two objects are: + +- A Deployment, describing a scalable group of identical pods. In this case, + you'll get just one replica, or copy of your pod. That pod, which is + described under `template`, has just one container in it. The container is + created from the image built by GitHub Actions in [Configure CI/CD for your + Go application](configure-ci-cd.md). +- A NodePort service, which will route traffic from port 30001 on your host to + port 8080 inside the pods it routes to, allowing you to reach your app + from the network. + +To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). + +### Deploy and check your application + +1. In a terminal, navigate to the project directory + and deploy your application to Kubernetes. + + ```console + $ kubectl apply -f docker-go-kubernetes.yaml + ``` + + You should see output that looks like the following, indicating your Kubernetes objects were created successfully. + + ```shell + deployment.apps/db created + service/db created + deployment.apps/server created + service/server created + ``` + +2. Make sure everything worked by listing your deployments. + + ```console + $ kubectl get deployments + ``` + + Your deployment should be listed as follows: + + ```shell + NAME READY UP-TO-DATE AVAILABLE AGE + db 1/1 1 1 76s + server 1/1 1 1 76s + ``` + + This indicates all of the pods are up and running. Do the same check for your services. + + ```console + $ kubectl get services + ``` + + You should get output like the following. + + ```shell + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + db ClusterIP 10.96.156.90 5432/TCP 2m8s + kubernetes ClusterIP 10.96.0.1 443/TCP 164m + server NodePort 10.102.94.225 8080:30001/TCP 2m8s + ``` + + In addition to the default `kubernetes` service, you can see your `server` service and `db` service. The `server` service is accepting traffic on port 30001/TCP. + +3. Open a terminal and curl your application to verify that it's working. + + ```console + $ curl --request POST \ + --url http://localhost:30001/send \ + --header 'content-type: application/json' \ + --data '{"value": "Hello, Oliver!"}' + ``` + + You should get the following message back. + + ```json + { "value": "Hello, Oliver!" } + ``` + +4. Run the following command to tear down your application. + + ```console + $ kubectl delete -f docker-go-kubernetes.yaml + ``` + +### Summary + +In this section, you learned how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. + +Related information: + +- [Kubernetes documentation](https://kubernetes.io/docs/home/) +- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) +- [Swarm mode overview](/manuals/engine/swarm/_index.md) diff --git a/content/guides/golang/build-images.md b/content/guides/golang/build-images.md deleted file mode 100644 index 2de5c21be3a0..000000000000 --- a/content/guides/golang/build-images.md +++ /dev/null @@ -1,489 +0,0 @@ ---- -title: Build your Go image -linkTitle: Build images -weight: 5 -keywords: containers, images, go, golang, dockerfiles, coding, build, push, run -description: Learn how to build your first Docker image by writing a Dockerfile -aliases: - - /get-started/golang/build-images/ - - /language/golang/build-images/ - - /guides/language/golang/build-images/ ---- - -## Overview - -In this section you're going to build a container image. The image includes -everything you need to run your application – the compiled application binary -file, the runtime, the libraries, and all other resources required by your -application. - -## Required software - -To complete this tutorial, you need the following: - -- Docker running locally. Follow the [instructions to download and install Docker](/manuals/desktop/_index.md). -- An IDE or a text editor to edit files. [Visual Studio Code](https://code.visualstudio.com/) is a free and popular choice but you can use anything you feel comfortable with. -- A Git client. This guide uses a command-line based `git` client, but you are free to use whatever works for you. -- A command-line terminal application. The examples shown in this module are from the Linux shell, but they should work in PowerShell, Windows Command Prompt, or OS X Terminal with minimal, if any, modifications. - -## Meet the example application - -The example application is a caricature of a microservice. It is purposefully trivial to keep focus on learning the basics of containerization for Go applications. - -The application offers two HTTP endpoints: - -- It responds with a string containing a heart symbol (`<3`) to requests to `/`. -- It responds with `{"Status" : "OK"}` JSON to a request to `/health`. - -It responds with HTTP error 404 to any other request. - -The application listens on a TCP port defined by the value of environment variable `PORT`. The default value is `8080`. - -The application is stateless. - -The complete source code for the application is on GitHub: [github.com/docker/docker-gs-ping](https://github.com/docker/docker-gs-ping). You are encouraged to fork it and experiment with it as much as you like. - -To continue, clone the application repository to your local machine: - -```console -$ git clone https://github.com/docker/docker-gs-ping -``` - -The application's `main.go` file is straightforward, if you are familiar with Go: - -```go -package main - -import ( - "net/http" - "os" - - "github.com/labstack/echo/v4" - "github.com/labstack/echo/v4/middleware" -) - -func main() { - - e := echo.New() - - e.Use(middleware.Logger()) - e.Use(middleware.Recover()) - - e.GET("/", func(c echo.Context) error { - return c.HTML(http.StatusOK, "Hello, Docker! <3") - }) - - e.GET("/health", func(c echo.Context) error { - return c.JSON(http.StatusOK, struct{ Status string }{Status: "OK"}) - }) - - httpPort := os.Getenv("PORT") - if httpPort == "" { - httpPort = "8080" - } - - e.Logger.Fatal(e.Start(":" + httpPort)) -} - -// Simple implementation of an integer minimum -// Adapted from: https://gobyexample.com/testing-and-benchmarking -func IntMin(a, b int) int { - if a < b { - return a - } - return b -} -``` - -## Create a Dockerfile for the application - -To build a container image with Docker, a `Dockerfile` with build instructions is required. - -Begin your `Dockerfile` with the (optional) parser directive line that instructs BuildKit to -interpret your file according to the grammar rules for the specified version of the syntax. - -You then tell Docker what base image you would like to use for your application: - -```dockerfile -# syntax=docker/dockerfile:1 - -FROM golang:1.19 -``` - -Docker images can be inherited from other images. Therefore, instead of creating -your own base image from scratch, you can use the official Go image that already -has all necessary tools and libraries to compile and run a Go application. - -> [!NOTE] -> -> If you are curious about creating your own base images, you can check out the following section of this guide: [creating base images](/manuals/build/building/base-images.md#create-a-base-image). -> Note, however, that this isn't necessary to continue with your task at hand. - -Now that you have defined the base image for your upcoming container image, you -can begin building on top of it. - -To make things easier when running the rest of your commands, create a directory -inside the image that you're building. This also instructs Docker to use this -directory as the default destination for all subsequent commands. This way you -don't have to type out full file paths in the `Dockerfile`, the relative paths -will be based on this directory. - -```dockerfile -WORKDIR /app -``` - -Usually the very first thing you do once you’ve downloaded a project written in -Go is to install the modules necessary to compile it. Note, that the base image -has the toolchain already, but your source code isn't in it yet. - -So before you can run `go mod download` inside your image, you need to get your -`go.mod` and `go.sum` files copied into it. Use the `COPY` command to do this. - -In its simplest form, the `COPY` command takes two parameters. The first -parameter tells Docker what files you want to copy into the image. The last -parameter tells Docker where you want that file to be copied to. - -Copy the `go.mod` and `go.sum` file into your project directory `/app` which, -owing to your use of `WORKDIR`, is the current directory (`./`) inside the -image. Unlike some modern shells that appear to be indifferent to the use of -trailing slash (`/`), and can figure out what the user meant (most of the time), -Docker's `COPY` command is quite sensitive in its interpretation of the trailing -slash. - -```dockerfile -COPY go.mod go.sum ./ -``` - -> [!NOTE] -> -> If you'd like to familiarize yourself with the trailing slash treatment by the -> `COPY` command, see [Dockerfile -> reference](/reference/dockerfile.md#copy). This trailing slash can -> cause issues in more ways than you can imagine. - -Now that you have the module files inside the Docker image that you are -building, you can use the `RUN` command to run the command `go mod download` -there as well. This works exactly the same as if you were running `go` locally -on your machine, but this time these Go modules will be installed into a -directory inside the image. - -```dockerfile -RUN go mod download -``` - -At this point, you have a Go toolchain version 1.19.x and all your Go -dependencies installed inside the image. - -The next thing you need to do is to copy your source code into the image. You’ll -use the `COPY` command just like you did with your module files before. - -```dockerfile -COPY *.go ./ -``` - -This `COPY` command uses a wildcard to copy all files with `.go` extension -located in the current directory on the host (the directory where the `Dockerfile` -is located) into the current directory inside the image. - -Now, to compile your application, use the familiar `RUN` command: - -```dockerfile -RUN CGO_ENABLED=0 GOOS=linux go build -o /docker-gs-ping -``` - -This should be familiar. The result of that command will be a static application -binary named `docker-gs-ping` and located in the root of the filesystem of the -image that you are building. You could have put the binary into any other place -you desire inside that image, the root directory has no special meaning in this -regard. It's convenient to use it to keep the file paths short for improved -readability. - -Now, all that is left to do is to tell Docker what command to run when your -image is used to start a container. - -You do this with the `CMD` command: - -```dockerfile -CMD ["/docker-gs-ping"] -``` - -Here's the complete `Dockerfile`: - -```dockerfile -# syntax=docker/dockerfile:1 - -FROM golang:1.19 - -# Set destination for COPY -WORKDIR /app - -# Download Go modules -COPY go.mod go.sum ./ -RUN go mod download - -# Copy the source code. Note the slash at the end, as explained in -# https://docs.docker.com/reference/dockerfile/#copy -COPY *.go ./ - -# Build -RUN CGO_ENABLED=0 GOOS=linux go build -o /docker-gs-ping - -# Optional: -# To bind to a TCP port, runtime parameters must be supplied to the docker command. -# But we can document in the Dockerfile what ports -# the application is going to listen on by default. -# https://docs.docker.com/reference/dockerfile/#expose -EXPOSE 8080 - -# Run -CMD ["/docker-gs-ping"] -``` - -The `Dockerfile` may also contain comments. They always begin with a `#` symbol, -and must be at the beginning of a line. Comments are there for your convenience -to allow documenting your `Dockerfile`. - -There is also a concept of Dockerfile directives, such as the `syntax` directive -you added. The directives must always be at the very top of the `Dockerfile`, so -when adding comments, make sure that the comments follow after any directives -that you may have used: - -```dockerfile -# syntax=docker/dockerfile:1 -# A sample microservice in Go packaged into a container image. - -FROM golang:1.19 - -# ... -``` - -## Build the image - -Now that you've created your `Dockerfile`, build an image from it. The `docker -build` command creates Docker images from the `Dockerfile` and a context. A -build context is the set of files located in the specified path or URL. The -Docker build process can access any of the files located in the context. - -The build command optionally takes a `--tag` flag. This flag is used to label -the image with a string value, which is easy for humans to read and recognize. -If you don't pass a `--tag`, Docker will use `latest` as the default value. - -Build your first Docker image. - -```console -$ docker build --tag docker-gs-ping . -``` - -The build process will print some diagnostic messages as it goes through the build steps. -The following is an example of what these messages may look like. - -```console -[+] Building 2.2s (15/15) FINISHED - => [internal] load build definition from Dockerfile 0.0s - => => transferring dockerfile: 701B 0.0s - => [internal] load .dockerignore 0.0s - => => transferring context: 2B 0.0s - => resolve image config for docker.io/docker/dockerfile:1 1.1s - => CACHED docker-image://docker.io/docker/dockerfile:1@sha256:39b85bbfa7536a5feceb7372a0817649ecb2724562a38360f4d6a7782a409b14 0.0s - => [internal] load build definition from Dockerfile 0.0s - => [internal] load .dockerignore 0.0s - => [internal] load metadata for docker.io/library/golang:1.19 0.7s - => [1/6] FROM docker.io/library/golang:1.19@sha256:5d947843dde82ba1df5ac1b2ebb70b203d106f0423bf5183df3dc96f6bc5a705 0.0s - => [internal] load build context 0.0s - => => transferring context: 6.08kB 0.0s - => CACHED [2/6] WORKDIR /app 0.0s - => CACHED [3/6] COPY go.mod go.sum ./ 0.0s - => CACHED [4/6] RUN go mod download 0.0s - => CACHED [5/6] COPY *.go ./ 0.0s - => CACHED [6/6] RUN CGO_ENABLED=0 GOOS=linux go build -o /docker-gs-ping 0.0s - => exporting to image 0.0s - => => exporting layers 0.0s - => => writing image sha256:ede8ff889a0d9bc33f7a8da0673763c887a258eb53837dd52445cdca7b7df7e3 0.0s - => => naming to docker.io/library/docker-gs-ping 0.0s -``` - -Your exact output will vary, but provided there aren't any errors, you should -see the word `FINISHED` in the first line of output. This means Docker has -successfully built your image named `docker-gs-ping`. - -## View local images - -To see the list of images you have on your local machine, you have two options. -One is to use the CLI and the other is to use [Docker -Desktop](/manuals/desktop/_index.md). Since you're working in the -terminal, take a look at listing images with the CLI. - -To list images, run the `docker image ls`command (or the `docker images` shorthand): - -```console -$ docker image ls - -REPOSITORY TAG IMAGE ID CREATED SIZE -docker-gs-ping latest 7f153fbcc0a8 2 minutes ago 1.11GB -... -``` - -Your exact output may vary, but you should see the `docker-gs-ping` image with -the `latest` tag. Because you didn't specify a custom tag when you built your -image, Docker assumed that the tag would be `latest`, which is a special value. - -## Tag images - -An image name is made up of slash-separated name components. Name components may -contain lowercase letters, digits, and separators. A separator is defined as a -period, one or two underscores, or one or more dashes. A name component may not -start or end with a separator. - -An image is made up of a manifest and a list of layers. In simple terms, a tag -points to a combination of these artifacts. You can have multiple tags for the -image and, in fact, most images have multiple tags. Create a second tag -for the image you built and take a look at its layers. - -Use the `docker image tag` (or `docker tag` shorthand) command to create a new -tag for your image. This command takes two arguments; the first argument is the -source image, and the second is the new tag to create. The following command -creates a new `docker-gs-ping:v1.0` tag for the `docker-gs-ping:latest` you -built: - -```console -$ docker image tag docker-gs-ping:latest docker-gs-ping:v1.0 -``` - -The Docker `tag` command creates a new tag for the image. It doesn't create a -new image. The tag points to the same image and is another way to reference -the image. - -Now run the `docker image ls` command again to see the updated list of local -images: - -```console -$ docker image ls - -REPOSITORY TAG IMAGE ID CREATED SIZE -docker-gs-ping latest 7f153fbcc0a8 6 minutes ago 1.11GB -docker-gs-ping v1.0 7f153fbcc0a8 6 minutes ago 1.11GB -... -``` - -You can see that you have two images that start with `docker-gs-ping`. You know -they're the same image because if you look at the `IMAGE ID` column, you can -see that the values are the same for the two images. This value is a unique -identifier Docker uses internally to identify the image. - -Remove the tag that you just created. To do this, you’ll use the -`docker image rm` command, or the shorthand `docker rmi` (which stands for -"remove image"): - -```console -$ docker image rm docker-gs-ping:v1.0 -Untagged: docker-gs-ping:v1.0 -``` - -Notice that the response from Docker tells you that the image hasn't been -removed but only untagged. - -Verify this by running the following command: - -```console -$ docker image ls -``` - -You will see that the tag `v1.0` is no longer in the list of images kept by your Docker instance. - -```text -REPOSITORY TAG IMAGE ID CREATED SIZE -docker-gs-ping latest 7f153fbcc0a8 7 minutes ago 1.11GB -... -``` - -The tag `v1.0` has been removed but you still have the `docker-gs-ping:latest` -tag available on your machine, so the image is there. - -## Multi-stage builds - -You may have noticed that your `docker-gs-ping` image weighs in at over a -gigabyte, which is a lot for a tiny compiled Go application. You may also be -wondering what happened to the full suite of Go tools, including the compiler, -after you had built your image. - -The answer is that the full toolchain is still there, in the container image. -Not only this is inconvenient because of the large file size, but it may also -present a security risk when the container is deployed. - -These two issues can be solved by using [multi-stage builds](/manuals/build/building/multi-stage.md). - -In a nutshell, a multi-stage build can carry over the artifacts from one build stage into another, -and every build stage can be instantiated from a different base image. - -Thus, in the following example, you are going to use a full-scale official Go -image to build your application. Then you'll copy the application binary into -another image whose base is very lean and doesn't include the Go toolchain or -other optional components. - -The `Dockerfile.multistage` in the sample application's repository has the -following content: - -```dockerfile -# syntax=docker/dockerfile:1 - -# Build the application from source -FROM golang:1.19 AS build-stage - -WORKDIR /app - -COPY go.mod go.sum ./ -RUN go mod download - -COPY *.go ./ - -RUN CGO_ENABLED=0 GOOS=linux go build -o /docker-gs-ping - -# Run the tests in the container -FROM build-stage AS run-test-stage -RUN go test -v ./... - -# Deploy the application binary into a lean image -FROM gcr.io/distroless/base-debian11 AS build-release-stage - -WORKDIR / - -COPY --from=build-stage /docker-gs-ping /docker-gs-ping - -EXPOSE 8080 - -USER nonroot:nonroot - -ENTRYPOINT ["/docker-gs-ping"] -``` - -Since you have two Dockerfiles now, you have to tell Docker what Dockerfile -you'd like to use to build the image. Tag the new image with `multistage`. This -tag (like any other, apart from `latest`) has no special meaning for Docker, -it's something you chose. - -```console -$ docker build -t docker-gs-ping:multistage -f Dockerfile.multistage . -``` - -Comparing the sizes of `docker-gs-ping:multistage` and `docker-gs-ping:latest` -you see a few orders-of-magnitude difference. - -```console -$ docker image ls -REPOSITORY TAG IMAGE ID CREATED SIZE -docker-gs-ping multistage e3fdde09f172 About a minute ago 28.1MB -docker-gs-ping latest 336a3f164d0f About an hour ago 1.11GB -``` - -This is so because the ["distroless"](https://github.com/GoogleContainerTools/distroless) -base image that you have used in the second stage of the build is very barebones and is designed for lean deployments of static binaries. - -There's much more to multi-stage builds, including the possibility of multi-architecture builds, -so feel free to check out [multi-stage builds](/manuals/build/building/multi-stage.md). This is, however, not essential for your progress here. - -## Next steps - -In this module, you met your example application and built and container image -for it. - -In the next module, you’ll take a look at how to run your image as a container. diff --git a/content/guides/golang/configure-ci-cd.md b/content/guides/golang/configure-ci-cd.md deleted file mode 100644 index 2f1f12a57c9d..000000000000 --- a/content/guides/golang/configure-ci-cd.md +++ /dev/null @@ -1,133 +0,0 @@ ---- -title: Configure CI/CD for your Go application -linkTitle: Configure CI/CD -weight: 40 -keywords: go, CI/CD, local, development -description: Learn how to Configure CI/CD for your Go application -aliases: - - /language/golang/configure-ci-cd/ - - /guides/language/golang/configure-ci-cd/ ---- - -## Prerequisites - -Complete the previous sections of this guide, starting with [Build your Go image](build-images.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. - -## Overview - -In this section, you'll learn how to set up and use GitHub Actions to build and push your Docker image to Docker Hub. You will complete the following steps: - -1. Create a new repository on GitHub. -2. Define the GitHub Actions workflow. -3. Run the workflow. - -## Step one: Create the repository - -Create a GitHub repository, configure the Docker Hub credentials, and push your source code. - -1. [Create a new repository](https://github.com/new) on GitHub. - -2. Open the repository **Settings**, and go to **Secrets and variables** > - **Actions**. - -3. Create a new **Repository variable** named `DOCKER_USERNAME` and your Docker ID as a value. - -4. Create a new [Personal Access Token (PAT)](/manuals/security/access-tokens.md#create-an-access-token) for Docker Hub. You can name this token `docker-tutorial`. Make sure access permissions include Read and Write. - -5. Add the PAT as a **Repository secret** in your GitHub repository, with the name - `DOCKERHUB_TOKEN`. - -6. In your local repository on your machine, run the following command to change - the origin to the repository you just created. Make sure you change - `your-username` to your GitHub username and `your-repository` to the name of - the repository you created. - - ```console - $ git remote set-url origin https://github.com/your-username/your-repository.git - ``` - -7. Run the following commands to stage, commit, and push your local repository to GitHub. - - ```console - $ git add -A - $ git commit -m "my commit" - $ git push -u origin main - ``` - -## Step two: Set up the workflow - -Set up your GitHub Actions workflow for building, testing, and pushing the image -to Docker Hub. - -1. Go to your repository on GitHub and then select the **Actions** tab. - -2. Select **set up a workflow yourself**. - - This takes you to a page for creating a new GitHub actions workflow file in - your repository, under `.github/workflows/main.yml` by default. - -3. In the editor window, copy and paste the following YAML configuration and commit the changes. - - ```yaml - name: ci - - on: - push: - branches: - - main - - jobs: - build: - runs-on: ubuntu-latest - steps: - - name: Login to Docker Hub - uses: docker/login-action@{{% param "login_action_version" %}} - with: - username: ${{ vars.DOCKER_USERNAME }} - password: ${{ secrets.DOCKERHUB_TOKEN }} - - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} - - - name: Build and push - uses: docker/build-push-action@{{% param "build_push_action_version" %}} - with: - platforms: linux/amd64,linux/arm64 - push: true - tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest - ``` - - For more information about the YAML syntax for `docker/build-push-action`, - refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). - -## Step three: Run the workflow - -Save the workflow file and run the job. - -1. Select **Commit changes...** and push the changes to the `main` branch. - - After pushing the commit, the workflow starts automatically. - -2. Go to the **Actions** tab. It displays the workflow. - - Selecting the workflow shows you the breakdown of all the steps. - -3. When the workflow is complete, go to your - [repositories on Docker Hub](https://hub.docker.com/repositories). - - If you see the new repository in that list, it means the GitHub Actions - successfully pushed the image to Docker Hub. - -## Summary - -In this section, you learned how to set up a GitHub Actions workflow for your application. - -Related information: - -- [Introduction to GitHub Actions](/guides/gha.md) -- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) -- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) - -## Next steps - -Next, learn how you can locally test and debug your workloads on Kubernetes before deploying. diff --git a/content/guides/golang/deploy.md b/content/guides/golang/deploy.md deleted file mode 100644 index 1002d6e3d1a9..000000000000 --- a/content/guides/golang/deploy.md +++ /dev/null @@ -1,249 +0,0 @@ ---- -title: Test your Go deployment -linkTitle: Test your deployment -weight: 50 -keywords: deploy, go, local, development -description: Learn how to deploy your Go application -aliases: - - /language/golang/deploy/ - - /guides/language/golang/deploy/ ---- - -## Prerequisites - -- Complete all the previous sections of this guide, starting with [Build - your Go image](build-images.md). -- [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker - Desktop. - -## Overview - -In this section, you'll learn how to use Docker Desktop to deploy your -application to a fully-featured Kubernetes environment on your development -machine. This allows you to test and debug your workloads on Kubernetes locally -before deploying. - -## Create a Kubernetes YAML file - -In your project directory, create a file named -`docker-go-kubernetes.yaml`. Open the file in an IDE or text editor and add -the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker -username and the name of the repository that you created in [Configure CI/CD for -your Go application](configure-ci-cd.md). - -```yaml -apiVersion: apps/v1 -kind: Deployment -metadata: - labels: - service: server - name: server - namespace: default -spec: - replicas: 1 - selector: - matchLabels: - service: server - strategy: {} - template: - metadata: - labels: - service: server - spec: - initContainers: - - name: wait-for-db - image: busybox:1.28 - command: - [ - "sh", - "-c", - 'until nc -zv db 5432; do echo "waiting for db"; sleep 2; done;', - ] - containers: - - env: - - name: PGDATABASE - value: mydb - - name: PGPASSWORD - value: whatever - - name: PGHOST - value: db - - name: PGPORT - value: "5432" - - name: PGUSER - value: postgres - image: DOCKER_USERNAME/REPO_NAME - name: server - imagePullPolicy: Always - ports: - - containerPort: 8080 - hostPort: 8080 - protocol: TCP - resources: {} - restartPolicy: Always -status: {} ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - labels: - service: db - name: db - namespace: default -spec: - replicas: 1 - selector: - matchLabels: - service: db - strategy: - type: Recreate - template: - metadata: - labels: - service: db - spec: - containers: - - env: - - name: POSTGRES_DB - value: mydb - - name: POSTGRES_PASSWORD - value: whatever - - name: POSTGRES_USER - value: postgres - image: postgres:18 - name: db - ports: - - containerPort: 5432 - protocol: TCP - resources: {} - restartPolicy: Always -status: {} ---- -apiVersion: v1 -kind: Service -metadata: - labels: - service: server - name: server - namespace: default -spec: - type: NodePort - ports: - - name: "8080" - port: 8080 - targetPort: 8080 - nodePort: 30001 - selector: - service: server -status: - loadBalancer: {} ---- -apiVersion: v1 -kind: Service -metadata: - labels: - service: db - name: db - namespace: default -spec: - ports: - - name: "5432" - port: 5432 - targetPort: 5432 - selector: - service: db -status: - loadBalancer: {} -``` - -In this Kubernetes YAML file, there are four objects, separated by the `---`. In addition to a Service and Deployment for the database, the other two objects are: - -- A Deployment, describing a scalable group of identical pods. In this case, - you'll get just one replica, or copy of your pod. That pod, which is - described under `template`, has just one container in it. The container is - created from the image built by GitHub Actions in [Configure CI/CD for your - Go application](configure-ci-cd.md). -- A NodePort service, which will route traffic from port 30001 on your host to - port 8080 inside the pods it routes to, allowing you to reach your app - from the network. - -To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). - -## Deploy and check your application - -1. In a terminal, navigate to the project directory - and deploy your application to Kubernetes. - - ```console - $ kubectl apply -f docker-go-kubernetes.yaml - ``` - - You should see output that looks like the following, indicating your Kubernetes objects were created successfully. - - ```shell - deployment.apps/db created - service/db created - deployment.apps/server created - service/server created - ``` - -2. Make sure everything worked by listing your deployments. - - ```console - $ kubectl get deployments - ``` - - Your deployment should be listed as follows: - - ```shell - NAME READY UP-TO-DATE AVAILABLE AGE - db 1/1 1 1 76s - server 1/1 1 1 76s - ``` - - This indicates all of the pods are up and running. Do the same check for your services. - - ```console - $ kubectl get services - ``` - - You should get output like the following. - - ```shell - NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE - db ClusterIP 10.96.156.90 5432/TCP 2m8s - kubernetes ClusterIP 10.96.0.1 443/TCP 164m - server NodePort 10.102.94.225 8080:30001/TCP 2m8s - ``` - - In addition to the default `kubernetes` service, you can see your `server` service and `db` service. The `server` service is accepting traffic on port 30001/TCP. - -3. Open a terminal and curl your application to verify that it's working. - - ```console - $ curl --request POST \ - --url http://localhost:30001/send \ - --header 'content-type: application/json' \ - --data '{"value": "Hello, Oliver!"}' - ``` - - You should get the following message back. - - ```json - { "value": "Hello, Oliver!" } - ``` - -4. Run the following command to tear down your application. - - ```console - $ kubectl delete -f docker-go-kubernetes.yaml - ``` - -## Summary - -In this section, you learned how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. - -Related information: - -- [Kubernetes documentation](https://kubernetes.io/docs/home/) -- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) -- [Swarm mode overview](/manuals/engine/swarm/_index.md) diff --git a/content/guides/golang/develop.md b/content/guides/golang/develop.md deleted file mode 100644 index d40e6ccd18c1..000000000000 --- a/content/guides/golang/develop.md +++ /dev/null @@ -1,733 +0,0 @@ ---- -title: Use containers for Go development -linkTitle: Develop your app -weight: 20 -keywords: get started, go, golang, local, development -description: Learn how to develop your application locally. -aliases: - - /get-started/golang/develop/ - - /language/golang/develop/ - - /guides/language/golang/develop/ ---- - -## Prerequisites - -Work through the steps of the [run your image as a container](run-containers.md) module to learn how to manage the lifecycle of your containers. - -## Introduction - -In this module, you'll take a look at running a database engine in a container and connecting it to the extended version of the example application. You are going to see some options for keeping persistent data and for wiring up the containers to talk to one another. Finally, you'll learn how to use Docker Compose to manage such multi-container local development environments effectively. - -## Local database and containers - -The database engine you are going to use is called [CockroachDB](https://www.cockroachlabs.com/product/). It is a modern, Cloud-native, distributed SQL database. - -Instead of compiling CockroachDB from the source code or using the operating system's native package manager to install CockroachDB, you are going to use the [Docker image for CockroachDB](https://hub.docker.com/r/cockroachdb/cockroach) and run it in a container. - -CockroachDB is compatible with PostgreSQL to a significant extent, and shares many conventions with the latter, particularly the default names for the environment variables. So, if you are familiar with Postgres, don't be surprised if you see some familiar environment variable names. The Go modules that work with Postgres, such as [pgx](https://pkg.go.dev/github.com/jackc/pgx), [pq](https://pkg.go.dev/github.com/lib/pq), [GORM](https://gorm.io/index.html), and [upper/db](https://upper.io/v4/) also work with CockroachDB. - -For more information on the relation between Go and CockroachDB, refer to the [CockroachDB documentation](https://www.cockroachlabs.com/docs/v20.2/build-a-go-app-with-cockroachdb.html), although this isn't necessary to continue with the present guide. - -### Storage - -The point of a database is to have a persistent store of data. [Volumes](/manuals/engine/storage/volumes.md) are the preferred mechanism for persisting data generated by and used by Docker containers. Thus, before you start CockroachDB, create the volume for it. - -To create a managed volume, run : - -```console -$ docker volume create roach -roach -``` - -You can view the list of all managed volumes in your Docker instance with the following command: - -```console -$ docker volume list -DRIVER VOLUME NAME -local roach -``` - -### Networking - -The example application and the database engine are going to talk to one another over the network. There are different kinds of network configuration possible, and you're going to use what's called a user-defined bridge network. It is going to provide you with a DNS lookup service so that you can refer to your database engine container by its hostname. - -The following command creates a new bridge network named `mynet`: - -```console -$ docker network create -d bridge mynet -51344edd6430b5acd121822cacc99f8bc39be63dd125a3b3cd517b6485ab7709 -``` - -As it was the case with the managed volumes, there is a command to list all networks set up in your Docker instance: - -```console -$ docker network list -NETWORK ID NAME DRIVER SCOPE -0ac2b1819fa4 bridge bridge local -51344edd6430 mynet bridge local -daed20bbecce host host local -6aee44f40a39 none null local -``` - -Your bridge network `mynet` has been created successfully. The other three networks, named `bridge`, `host`, and `none` are the default networks and they had been created by the Docker itself. While it's not relevant to this guide, you can learn more about Docker networking in the [networking overview](/manuals/engine/network/_index.md) section. - -### Choose good names for volumes and networks - -As the saying goes, there are only two hard things in Computer Science: cache invalidation and naming things. And off-by-one errors. - -When choosing a name for a network or a managed volume, it's best to choose a name that's indicative of the intended purpose. This guide aims for brevity, so it used short, generic names. - -### Start the database engine - -Now that the housekeeping chores are done, you can run CockroachDB in a container and attach it to the volume and network you had just created. When you run the following command, Docker will pull the image from Docker Hub and run it for you locally: - -```console -$ docker run -d \ - --name roach \ - --hostname db \ - --network mynet \ - -p 26257:26257 \ - -p 8080:8080 \ - -v roach:/cockroach/cockroach-data \ - cockroachdb/cockroach:latest-v25.4 start-single-node \ - --insecure - -# ... output omitted ... -``` - -Notice a clever use of the tag `latest-v25.4` to make sure that you're pulling the latest patch version of 25.4. The diversity of available tags depend on the image maintainer. Here, your intent was to have the latest patched version of CockroachDB while not straying too far away from the known working version as the time goes by. To see the tags available for the CockroachDB image, you can go to the [CockroachDB page on Docker Hub](https://hub.docker.com/r/cockroachdb/cockroach/tags). - -### Configure the database engine - -Now that the database engine is live, there is some configuration to do before your application can begin using it. Fortunately, it's not a lot. You must: - -1. Create a blank database. -2. Register a new user account with the database engine. -3. Grant that new user access rights to the database. - -You can do that with the help of CockroachDB built-in SQL shell. To start the SQL shell in the same container where the database engine is running, type: - -```console -$ docker exec -it roach ./cockroach sql --insecure -``` - -1. In the SQL shell, create the database that the example application is going to use: - - ```sql - CREATE DATABASE mydb; - ``` - -2. Register a new SQL user account with the database engine. Use the username `totoro`. - - ```sql - CREATE USER totoro; - ``` - -3. Give the new user the necessary permissions: - - ```sql - GRANT ALL ON DATABASE mydb TO totoro; - ``` - -4. Type `quit` to exit the shell. - -The following is an example of interaction with the SQL shell. - -```console -$ sudo docker exec -it roach ./cockroach sql --insecure -# -# Welcome to the CockroachDB SQL shell. -# All statements must be terminated by a semicolon. -# To exit, type: \q. -# -# Server version: CockroachDB CCL v20.1.15 (x86_64-unknown-linux-gnu, built 2021/04/26 16:11:58, go1.13.9) (same version as client) -# Cluster ID: 7f43a490-ccd6-4c2a-9534-21f393ca80ce -# -# Enter \? for a brief introduction. -# -root@:26257/defaultdb> CREATE DATABASE mydb; -CREATE DATABASE - -Time: 22.985478ms - -root@:26257/defaultdb> CREATE USER totoro; -CREATE ROLE - -Time: 13.921659ms - -root@:26257/defaultdb> GRANT ALL ON DATABASE mydb TO totoro; -GRANT - -Time: 14.217559ms - -root@:26257/defaultdb> quit -oliver@hki:~$ -``` - -### Meet the example application - -Now that you have started and configured the database engine, you can switch your attention to the application. - -The example application for this module is an extended version of `docker-gs-ping` application you've used in the previous modules. You have two options: - -- You can update your local copy of `docker-gs-ping` to match the new extended version presented in this chapter; or -- You can clone the [docker/docker-gs-ping-dev](https://github.com/docker/docker-gs-ping-dev) repository. This latter approach is recommended. - -To checkout the example application, run: - -```console -$ git clone https://github.com/docker/docker-gs-ping-dev.git -# ... output omitted ... -``` - -The application's `main.go` now includes database initialization code, as well as the code to implement a new business requirement: - -- An HTTP `POST` request to `/send` containing a `{ "value" : string }` JSON must save the value to the database. - -You also have an update for another business requirement. The requirement was: - -- The application responds with a text message containing a heart symbol ("`<3`") on requests to `/`. - -And now it's going to be: - -- The application responds with the string containing the count of messages stored in the database, enclosed in the parentheses. - - Example output: `Hello, Docker! (7)` - -The full source code listing of `main.go` follows. - -```go -package main - -import ( - "context" - "database/sql" - "fmt" - "log" - "net/http" - "os" - - "github.com/cenkalti/backoff/v4" - "github.com/cockroachdb/cockroach-go/v2/crdb" - "github.com/labstack/echo/v4" - "github.com/labstack/echo/v4/middleware" -) - -func main() { - - e := echo.New() - - e.Use(middleware.Logger()) - e.Use(middleware.Recover()) - - db, err := initStore() - if err != nil { - log.Fatalf("failed to initialize the store: %s", err) - } - defer db.Close() - - e.GET("/", func(c echo.Context) error { - return rootHandler(db, c) - }) - - e.GET("/ping", func(c echo.Context) error { - return c.JSON(http.StatusOK, struct{ Status string }{Status: "OK"}) - }) - - e.POST("/send", func(c echo.Context) error { - return sendHandler(db, c) - }) - - httpPort := os.Getenv("HTTP_PORT") - if httpPort == "" { - httpPort = "8080" - } - - e.Logger.Fatal(e.Start(":" + httpPort)) -} - -type Message struct { - Value string `json:"value"` -} - -func initStore() (*sql.DB, error) { - - pgConnString := fmt.Sprintf("host=%s port=%s dbname=%s user=%s password=%s sslmode=disable", - os.Getenv("PGHOST"), - os.Getenv("PGPORT"), - os.Getenv("PGDATABASE"), - os.Getenv("PGUSER"), - os.Getenv("PGPASSWORD"), - ) - - var ( - db *sql.DB - err error - ) - openDB := func() error { - db, err = sql.Open("postgres", pgConnString) - return err - } - - err = backoff.Retry(openDB, backoff.NewExponentialBackOff()) - if err != nil { - return nil, err - } - - if _, err := db.Exec( - "CREATE TABLE IF NOT EXISTS message (value TEXT PRIMARY KEY)"); err != nil { - return nil, err - } - - return db, nil -} - -func rootHandler(db *sql.DB, c echo.Context) error { - r, err := countRecords(db) - if err != nil { - return c.HTML(http.StatusInternalServerError, err.Error()) - } - return c.HTML(http.StatusOK, fmt.Sprintf("Hello, Docker! (%d)\n", r)) -} - -func sendHandler(db *sql.DB, c echo.Context) error { - - m := &Message{} - - if err := c.Bind(m); err != nil { - return c.JSON(http.StatusInternalServerError, err) - } - - err := crdb.ExecuteTx(context.Background(), db, nil, - func(tx *sql.Tx) error { - _, err := tx.Exec( - "INSERT INTO message (value) VALUES ($1) ON CONFLICT (value) DO UPDATE SET value = excluded.value", - m.Value, - ) - if err != nil { - return c.JSON(http.StatusInternalServerError, err) - } - return nil - }) - - if err != nil { - return c.JSON(http.StatusInternalServerError, err) - } - - return c.JSON(http.StatusOK, m) -} - -func countRecords(db *sql.DB) (int, error) { - - rows, err := db.Query("SELECT COUNT(*) FROM message") - if err != nil { - return 0, err - } - defer rows.Close() - - count := 0 - for rows.Next() { - if err := rows.Scan(&count); err != nil { - return 0, err - } - rows.Close() - } - - return count, nil -} -``` - -The repository also includes the `Dockerfile`, which is almost exactly the same as the multi-stage `Dockerfile` introduced in the previous modules. It uses the official Docker Go image to build the application and then builds the final image by placing the compiled binary into the much slimmer, distroless image. - -Regardless of whether you had updated the old example application, or checked out the new one, this new Docker image has to be built to reflect the changes to the application source code. - -### Build the application - -You can build the image with the familiar `build` command: - -```console -$ docker build --tag docker-gs-ping-roach . -``` - -### Run the application - -Now, run your container. This time you'll need to set some environment variables so that your application knows how to access the database. For now, you’ll do this right in the `docker run` command. Later you'll see a more convenient method with Docker Compose. - -> [!NOTE] -> -> Since you're running your CockroachDB cluster in insecure mode, the value for the password can be anything. -> -> In production, don't run in insecure mode. - -```console -$ docker run -it --rm -d \ - --network mynet \ - --name rest-server \ - -p 80:8080 \ - -e PGUSER=totoro \ - -e PGPASSWORD=myfriend \ - -e PGHOST=db \ - -e PGPORT=26257 \ - -e PGDATABASE=mydb \ - docker-gs-ping-roach -``` - -There are a few points to note about this command. - -- You map container port `8080` to host port `80` this time. Thus, for `GET` requests you can get away with literally `curl localhost`: - - ```console - $ curl localhost - Hello, Docker! (0) - ``` - - Or, if you prefer, a proper URL would work just as well: - - ```console - $ curl http://localhost/ - Hello, Docker! (0) - ``` - -- The total number of stored messages is `0` for now. This is fine, because you haven't posted anything to your application yet. -- You refer to the database container by its hostname, which is `db`. This is why you had `--hostname db` when you started the database container. - -- The actual password doesn't matter, but it must be set to something to avoid confusing the example application. -- The container you've just run is named `rest-server`. These names are useful for managing the container lifecycle: - - ```console - # Don't do this just yet, it's only an example: - $ docker container rm --force rest-server - ``` - -### Test the application - -In the previous section, you've already tested querying your application with `GET` and it returned zero for the stored message counter. Now, post some messages to it: - -```console -$ curl --request POST \ - --url http://localhost/send \ - --header 'content-type: application/json' \ - --data '{"value": "Hello, Docker!"}' -``` - -The application responds with the contents of the message, which means it has been saved in the database: - -```json -{ "value": "Hello, Docker!" } -``` - -Send another message: - -```console -$ curl --request POST \ - --url http://localhost/send \ - --header 'content-type: application/json' \ - --data '{"value": "Hello, Oliver!"}' -``` - -And again, you get the value of the message back: - -```json -{ "value": "Hello, Oliver!" } -``` - -Run curl and see what the message counter says: - -```console -$ curl localhost -Hello, Docker! (2) -``` - -In this example, you sent two messages and the database kept them. Or has it? Stop and remove all your containers, but not the volumes, and try again. - -First, stop the containers: - -```console -$ docker container stop rest-server roach -rest-server -roach -``` - -Then, remove them: - -```console -$ docker container rm rest-server roach -rest-server -roach -``` - -Verify that they're gone: - -```console -$ docker container list --all -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -``` - -And start them again, database first: - -```console -$ docker run -d \ - --name roach \ - --hostname db \ - --network mynet \ - -p 26257:26257 \ - -p 8080:8080 \ - -v roach:/cockroach/cockroach-data \ - cockroachdb/cockroach:latest-v25.4 start-single-node \ - --insecure -``` - -And the service next: - -```console -$ docker run -it --rm -d \ - --network mynet \ - --name rest-server \ - -p 80:8080 \ - -e PGUSER=totoro \ - -e PGPASSWORD=myfriend \ - -e PGHOST=db \ - -e PGPORT=26257 \ - -e PGDATABASE=mydb \ - docker-gs-ping-roach -``` - -Lastly, query your service: - -```console -$ curl localhost -Hello, Docker! (2) -``` - -Great! The count of records from the database is correct although you haven't only stopped the containers, but you've also removed them before starting new instances. The difference is in the managed volume for CockroachDB, which you reused. The new CockroachDB container has read the database files from the disk, just as it normally would if it were running outside the container. - -### Wind down everything - -Remember, that you're running CockroachDB in insecure mode. Now that you've built and tested your application, it's time to wind everything down before moving on. You can list the containers that you are running with the `list` command: - -```console -$ docker container list -``` - -Now that you know the container IDs, you can use `docker container stop` and `docker container rm`, as demonstrated in the previous modules. - -Stop the CockroachDB and `docker-gs-ping-roach` containers before moving on. - -## Better productivity with Docker Compose - -At this point, you might be wondering if there is a way to avoid having to deal with long lists of arguments to the `docker` command. The toy example you used in this series requires five environment variables to define the connection to the database. A real application might need many, many more. Then there is also a question of dependencies. Ideally, you want to make sure that the database is started before your application is run. And spinning up the database instance may require another Docker command with many options. But there is a better way to orchestrate these deployments for local development purposes. - -In this section, you'll create a Docker Compose file to start your `docker-gs-ping-roach` application and CockroachDB database engine with a single command. - -### Configure Docker Compose - -In your application's directory, create a new text file named `compose.yaml` with the following content. - -```yaml -services: - docker-gs-ping-roach: - depends_on: - - roach - build: - context: . - container_name: rest-server - hostname: rest-server - networks: - - mynet - ports: - - 80:8080 - environment: - - PGUSER=${PGUSER:-totoro} - - PGPASSWORD=${PGPASSWORD:?database password not set} - - PGHOST=${PGHOST:-db} - - PGPORT=${PGPORT:-26257} - - PGDATABASE=${PGDATABASE:-mydb} - deploy: - restart_policy: - condition: on-failure - roach: - image: cockroachdb/cockroach:latest-v25.4 - container_name: roach - hostname: db - networks: - - mynet - ports: - - 26257:26257 - - 8080:8080 - volumes: - - roach:/cockroach/cockroach-data - command: start-single-node --insecure - -volumes: - roach: - -networks: - mynet: - driver: bridge -``` - -This Docker Compose configuration is super convenient as you don't have to type all the parameters to pass to the `docker run` command. You can declaratively do that in the Docker Compose file. The [Docker Compose documentation pages](/manuals/compose/_index.md) are quite extensive and include a full reference for the Docker Compose file format. - -### The `.env` file - -Docker Compose will automatically read environment variables from a `.env` file if it's available. Since your Compose file requires `PGPASSWORD` to be set, add the following content to the `.env` file: - -```bash -PGPASSWORD=whatever -``` - -The exact value doesn't really matter for this example, because you run CockroachDB in insecure mode. Make sure you set the variable to some value to avoid getting an error. - -### Merging Compose files - -The filename `compose.yaml` is the default filename which `docker compose` command recognizes if no `-f` flag is provided. This means you can have multiple Docker Compose files if your environment has such requirements. Furthermore, Docker Compose files are composable, so multiple files can be specified on the command line to merge parts of the configuration together. The following list shows a few examples of scenarios where such a feature would be useful: - -- Using a bind mount for the source code for local development but not when running the CI tests; -- Switching between using a pre-built image for the frontend for some API application vs creating a bind mount for source code; -- Adding additional services for integration testing; -- And many more... - -You aren't going to cover any of these advanced use cases here. - -### Variable substitution in Docker Compose - -One of the really cool features of Docker Compose is [variable substitution](/reference/compose-file/interpolation.md). You can see some examples in the Compose file, `environment` section. By means of an example: - -- `PGUSER=${PGUSER:-totoro}` means that inside the container, the environment variable `PGUSER` shall be set to the same value as it has on the host machine where Docker Compose is run. If there is no environment variable with this name on the host machine, the variable inside the container gets the default value of `totoro`. -- `PGPASSWORD=${PGPASSWORD:?database password not set}` means that if the environment variable `PGPASSWORD` isn't set on the host, Docker Compose will display an error. This is OK, because you don't want to hard-code default values for the password. You set the password value in the `.env` file, which is local to your machine. It is always a good idea to add `.env` to `.gitignore` to prevent the secrets being checked into the version control. - -Other ways of dealing with undefined or empty values exist, as documented in the [variable substitution](/reference/compose-file/interpolation.md) section of the Docker documentation. - -### Validating Docker Compose configuration - -Before you apply changes made to a Compose configuration file, there is an opportunity to validate the content of the configuration file with the following command: - -```console -$ docker compose config -``` - -When this command is run, Docker Compose reads the file `compose.yaml`, parses it into a data structure in memory, validates where possible, and prints back the reconstruction of that configuration file from its internal representation. If this isn't possible due to errors, Docker prints an error message instead. - -### Build and run the application using Docker Compose - -Start your application and confirm that it's running. - -```console -$ docker compose up --build -``` - -You passed the `--build` flag so Docker will compile your image and then start it. - -> [!NOTE] -> -> Docker Compose is a useful tool, but it has its own quirks. For example, no rebuild is triggered on the update to the source code unless the `--build` flag is provided. It is a very common pitfall to edit one's source code, and forget to use the `--build` flag when running `docker compose up`. - -Since your set-up is now run by Docker Compose, it has assigned it a project name, so you get a new volume for your CockroachDB instance. This means that your application will fail to connect to the database, because the database doesn't exist in this new volume. The terminal displays an authentication error for the database: - -```text -# ... omitted output ... -rest-server | 2021/05/10 00:54:25 failed to initialise the store: pq: password authentication failed for user totoro -roach | * -roach | * INFO: Replication was disabled for this cluster. -roach | * When/if adding nodes in the future, update zone configurations to increase the replication factor. -roach | * -roach | CockroachDB node starting at 2021-05-10 00:54:26.398177 +0000 UTC (took 3.0s) -roach | build: CCL v20.1.15 @ 2021/04/26 16:11:58 (go1.13.9) -roach | webui: http://db:8080 -roach | sql: postgresql://root@db:26257?sslmode=disable -roach | RPC client flags: /cockroach/cockroach --host=db:26257 --insecure -roach | logs: /cockroach/cockroach-data/logs -roach | temp dir: /cockroach/cockroach-data/cockroach-temp349434348 -roach | external I/O path: /cockroach/cockroach-data/extern -roach | store[0]: path=/cockroach/cockroach-data -roach | storage engine: rocksdb -roach | status: initialized new cluster -roach | clusterID: b7b1cb93-558f-4058-b77e-8a4ddb329a88 -roach | nodeID: 1 -rest-server exited with code 0 -rest-server | 2021/05/10 00:54:25 failed to initialise the store: pq: password authentication failed for user totoro -rest-server | 2021/05/10 00:54:26 failed to initialise the store: pq: password authentication failed for user totoro -rest-server | 2021/05/10 00:54:29 failed to initialise the store: pq: password authentication failed for user totoro -rest-server | 2021/05/10 00:54:25 failed to initialise the store: pq: password authentication failed for user totoro -rest-server | 2021/05/10 00:54:26 failed to initialise the store: pq: password authentication failed for user totoro -rest-server | 2021/05/10 00:54:29 failed to initialise the store: pq: password authentication failed for user totoro -rest-server exited with code 1 -# ... omitted output ... -``` - -Because of the way you set up your deployment using `restart_policy`, the failing container is being restarted every 20 seconds. So, in order to fix the problem, you need to log in to the database engine and create the user. You've done it before in [Configure the database engine](#configure-the-database-engine). - -This isn't a big deal. All you have to do is to connect to CockroachDB instance and run the three SQL commands to create the database and the user, as described in [Configure the database engine](#configure-the-database-engine). - -So, log in to the database engine from another terminal: - -```console -$ docker exec -it roach ./cockroach sql --insecure -``` - -And run the same commands as before to create the database `mydb`, the user `totoro`, and to grant that user necessary permissions. Once you do that (and the example application container is automatically restarts), the `rest-service` stops failing and restarting and the console goes quiet. - -It would have been possible to connect the volume that you had previously used, but for the purposes of this example it's more trouble than it's worth and it also provided an opportunity to show how to introduce resilience into your deployment via the `restart_policy` Compose file feature. - -### Testing the application - -Now, test your API endpoint. In the new terminal, run the following command: - -```console -$ curl http://localhost/ -``` - -You should receive the following response: - -```json -Hello, Docker! (0) -``` - -### Shutting down - -To stop the containers started by Docker Compose, press `ctrl+c` in the terminal where you ran `docker compose up`. To remove those containers after they've been stopped, run `docker compose down`. - -### Detached mode - -You can run containers started by the `docker compose` command in detached mode, just as you would with the `docker` command, by using the `-d` flag. - -To start the stack, defined by the Compose file in detached mode, run: - -```console -$ docker compose up --build -d -``` - -Then, you can use `docker compose stop` to stop the containers and `docker compose down` to remove them. - -## Further exploration - -You can run `docker compose` to see what other commands are available. - -## Wrap up - -There are some tangential, yet interesting points that were purposefully not covered in this chapter. For the more adventurous reader, this section offers some pointers for further study. - -### Persistent storage - -A managed volume isn't the only way to provide your container with persistent storage. It is highly recommended to get acquainted with available storage options and their use cases, covered in [Manage data in Docker](/manuals/engine/storage/_index.md). - -### CockroachDB clusters - -You ran a single instance of CockroachDB, which was enough for this example. But, it's possible to run a CockroachDB cluster, which is made of multiple instances of CockroachDB, each instance running in its own container. Since CockroachDB engine is distributed by design, it would have taken surprisingly little change to your procedure to run a cluster with multiple nodes. - -Such distributed set-up offers interesting possibilities, such as applying Chaos Engineering techniques to simulate parts of the cluster failing and evaluating your application's ability to cope with such failures. - -If you are interested in experimenting with CockroachDB clusters, check out: - -- [Start a CockroachDB Cluster in Docker](https://www.cockroachlabs.com/docs/v20.2/start-a-local-cluster-in-docker-mac.html) article; and -- Documentation for Docker Compose keywords [`deploy`](/reference/compose-file/legacy-versions.md) and [`replicas`](/reference/compose-file/legacy-versions.md). - -### Other databases - -Since you didn't run a cluster of CockroachDB instances, you might be wondering whether you could have used a non-distributed database engine. The answer is 'yes', and if you were to pick a more traditional SQL database, such as [PostgreSQL](https://www.postgresql.org/), the process described in this chapter would have been very similar. - -## Next steps - -In this module, you set up a containerized development environment with your application and the database engine running in different containers. You also wrote a Docker Compose file which links the two containers together and provides for easy starting up and tearing down of the development environment. - -In the next module, you'll take a look at one possible approach to running functional tests in Docker. diff --git a/content/guides/golang/run-containers.md b/content/guides/golang/run-containers.md deleted file mode 100644 index 34e87c648915..000000000000 --- a/content/guides/golang/run-containers.md +++ /dev/null @@ -1,210 +0,0 @@ ---- -title: Run your Go image as a container -linkTitle: Run containers -weight: 10 -keywords: get started, go, golang, run, container -description: Learn how to run the image as a container. -aliases: - - /get-started/golang/run-containers/ - - /language/golang/run-containers/ - - /guides/language/golang/run-containers/ ---- - -## Prerequisites - -Work through the steps to containerize a Go application in [Build your Go image](build-images.md). - -## Overview - -In the previous module you created a `Dockerfile` for your example application and then you created your Docker image using the command `docker build`. Now that you have the image, you can run that image and see if your application is running correctly. - -A container is a normal operating system process except that this process is isolated and has its own file system, its own networking, and its own isolated process tree separate from the host. - -To run an image inside of a container, you use the `docker run` command. It requires one parameter and that's the image name. Start your image and make sure it's running correctly. Run the following command in your terminal. - -```console -$ docker run docker-gs-ping -``` - -```text - ____ __ - / __/___/ / ___ - / _// __/ _ \/ _ \ -/___/\__/_//_/\___/ v4.10.2 -High performance, minimalist Go web framework -https://echo.labstack.com -____________________________________O/_______ - O\ -⇨ http server started on [::]:8080 -``` - -When you run this command, you’ll notice that you weren't returned to the command prompt. This is because your application is a REST server and will run in a loop waiting for incoming requests without returning control back to the OS until you stop the container. - -Make a GET request to the server using the curl command. - -```console -$ curl http://localhost:8080/ -curl: (7) Failed to connect to localhost port 8080: Connection refused -``` - -Your curl command failed because the connection to your server was refused. -Meaning that you weren't able to connect to localhost on port 8080. This is -expected because your container is running in isolation which includes -networking. Stop the container and restart with port 8080 published on your -local network. - -To stop the container, press ctrl-c. This will return you to the terminal prompt. - -To publish a port for your container, you’ll use the `--publish` flag (`-p` for short) on the `docker run` command. The format of the `--publish` command is `[host_port]:[container_port]`. So if you wanted to expose port `8080` inside the container to port `3000` outside the container, you would pass `3000:8080` to the `--publish` flag. - -Start the container and expose port `8080` to port `8080` on the host. - -```console -$ docker run --publish 8080:8080 docker-gs-ping -``` - -Now, rerun the curl command. - -```console -$ curl http://localhost:8080/ -Hello, Docker! <3 -``` - -Success! You were able to connect to the application running inside of your container on port 8080. Switch back to the terminal where your container is running and you should see the `GET` request logged to the console. - -Press `ctrl-c` to stop the container. - -## Run in detached mode - -This is great so far, but your sample application is a web server and you -shouldn't have to have your terminal connected to the container. Docker can run -your container in detached mode in the background. To do this, you can use the -`--detach` or `-d` for short. Docker will start your container the same as -before but this time will detach from the container and return you to the -terminal prompt. - -```console -$ docker run -d -p 8080:8080 docker-gs-ping -d75e61fcad1e0c0eca69a3f767be6ba28a66625ce4dc42201a8a323e8313c14e -``` - -Docker started your container in the background and printed the container ID on the terminal. - -Again, make sure that your container is running. Run the same `curl` command: - -```console -$ curl http://localhost:8080/ -Hello, Docker! <3 -``` - -## List containers - -Since you ran your container in the background, how do you know if your container is running or what other containers are running on your machine? Well, to see a list of containers running on your machine, run `docker ps`. This is similar to how the ps command is used to see a list of processes on a Linux machine. - -```console -$ docker ps - -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -d75e61fcad1e docker-gs-ping "/docker-gs-ping" 41 seconds ago Up 40 seconds 0.0.0.0:8080->8080/tcp inspiring_ishizaka -``` - -The `ps` command tells you a bunch of stuff about your running containers. You can see the container ID, the image running inside the container, the command that was used to start the container, when it was created, the status, ports that are exposed, and the names of the container. - -You are probably wondering where the name of your container is coming from. Since you didn’t provide a name for the container when you started it, Docker generated a random name. You'll fix this in a minute but first you need to stop the container. To stop the container, run the `docker stop` command, passing the container's name or ID. - -```console -$ docker stop inspiring_ishizaka -inspiring_ishizaka -``` - -Now rerun the `docker ps` command to see a list of running containers. - -```console -$ docker ps - -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -``` - -## Stop, start, and name containers - -Docker containers can be started, stopped and restarted. When you stop a container, it's not removed but the status is changed to stopped and the process inside of the container is stopped. When you ran the `docker ps` command, the default output is to only show running containers. If you pass the `--all` or `-a` for short, you will see all containers on your system, including stopped containers and running containers. - -```console -$ docker ps --all - -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -d75e61fcad1e docker-gs-ping "/docker-gs-ping" About a minute ago Exited (2) 23 seconds ago inspiring_ishizaka -f65dbbb9a548 docker-gs-ping "/docker-gs-ping" 3 minutes ago Exited (2) 2 minutes ago wizardly_joliot -aade1bf3d330 docker-gs-ping "/docker-gs-ping" 3 minutes ago Exited (2) 3 minutes ago magical_carson -52d5ce3c15f0 docker-gs-ping "/docker-gs-ping" 9 minutes ago Exited (2) 3 minutes ago gifted_mestorf -``` - -If you’ve been following along, you should see several containers listed. These are containers that you started and stopped but haven't removed yet. - -Restart the container that you have just stopped. Locate the name of the container and replace the name of the container in the following `restart` command: - -```console -$ docker restart inspiring_ishizaka -``` - -Now, list all the containers again using the `ps` command: - -```console -$ docker ps -a - -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -d75e61fcad1e docker-gs-ping "/docker-gs-ping" 2 minutes ago Up 5 seconds 0.0.0.0:8080->8080/tcp inspiring_ishizaka -f65dbbb9a548 docker-gs-ping "/docker-gs-ping" 4 minutes ago Exited (2) 2 minutes ago wizardly_joliot -aade1bf3d330 docker-gs-ping "/docker-gs-ping" 4 minutes ago Exited (2) 4 minutes ago magical_carson -52d5ce3c15f0 docker-gs-ping "/docker-gs-ping" 10 minutes ago Exited (2) 4 minutes ago gifted_mestorf -``` - -Notice that the container you just restarted has been started in detached mode and has port `8080` exposed. Also, note that the status of the container is `Up X seconds`. When you restart a container, it will be started with the same flags or commands that it was originally started with. - -Stop and remove all of your containers and take a look at fixing the random naming issue. - -Stop the container you just started. Find the name of your running container and replace the name in the following command with the name of the container on your system: - -```console -$ docker stop inspiring_ishizaka -inspiring_ishizaka -``` - -Now that all of your containers are stopped, remove them. When a container is removed, it's no longer running nor is it in the stopped state. Instead, the process inside the container is terminated and the metadata for the container is removed. - -To remove a container, run the `docker rm` command passing the container name. You can pass multiple container names to the command in one command. - -Again, make sure you replace the containers names in the following command with the container names from your system: - -```console -$ docker rm inspiring_ishizaka wizardly_joliot magical_carson gifted_mestorf - -inspiring_ishizaka -wizardly_joliot -magical_carson -gifted_mestorf -``` - -Run the `docker ps --all` command again to verify that all containers are gone. - -Now, address the pesky random name issue. Standard practice is to name your containers for the simple reason that it's easier to identify what's running in the container and what application or service it's associated with. Just like good naming conventions for variables in your code makes it simpler to read. So goes naming your containers. - -To name a container, you must pass the `--name` flag to the `run` command: - -```console -$ docker run -d -p 8080:8080 --name rest-server docker-gs-ping -3bbc6a3102ea368c8b966e1878a5ea9b1fc61187afaac1276c41db22e4b7f48f -``` - -```console -$ docker ps - -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -3bbc6a3102ea docker-gs-ping "/docker-gs-ping" 25 seconds ago Up 24 seconds 0.0.0.0:8080->8080/tcp rest-server -``` - -Now, you can easily identify your container based on the name. - -## Next steps - -In this module, you learned how to run containers and publish ports. You also learned to manage the lifecycle of containers. You then learned the importance of naming your containers so that they're more easily identifiable. In the next module, you’ll learn how to run a database in a container and connect it to your application. diff --git a/content/guides/golang/run-tests.md b/content/guides/golang/run-tests.md deleted file mode 100644 index b4b435fd62e2..000000000000 --- a/content/guides/golang/run-tests.md +++ /dev/null @@ -1,96 +0,0 @@ ---- -title: Run your tests using Go test -linkTitle: Run your tests -weight: 30 -keywords: build, go, golang, test -description: How to build and run your Go tests in a container -aliases: - - /get-started/golang/run-tests/ - - /language/golang/run-tests/ - - /guides/language/golang/run-tests/ ---- - -## Prerequisites - -Complete the [Build your Go image](build-images.md) section of this guide. - -## Overview - -Testing is an essential part of modern software development. Testing can mean a -lot of things to different development teams. There are unit tests, integration -tests and end-to-end testing. In this guide you take a look at running your unit -tests in Docker when building. - -For this section, use the `docker-gs-ping` project that you cloned in [Build -your Go image](build-images.md). - -## Run tests when building - -To run your tests when building, you need to add a test stage to the -`Dockerfile.multistage`. The `Dockerfile.multistage` in the sample application's -repository already has the following content: - -```dockerfile {hl_lines="15-17"} -# syntax=docker/dockerfile:1 - -# Build the application from source -FROM golang:1.19 AS build-stage - -WORKDIR /app - -COPY go.mod go.sum ./ -RUN go mod download - -COPY *.go ./ - -RUN CGO_ENABLED=0 GOOS=linux go build -o /docker-gs-ping - -# Run the tests in the container -FROM build-stage AS run-test-stage -RUN go test -v ./... - -# Deploy the application binary into a lean image -FROM gcr.io/distroless/base-debian11 AS build-release-stage - -WORKDIR / - -COPY --from=build-stage /docker-gs-ping /docker-gs-ping - -EXPOSE 8080 - -USER nonroot:nonroot - -ENTRYPOINT ["/docker-gs-ping"] -``` - -Run the following command to build an image using the `run-test-stage` stage as the target and view the test results. Include `--progress plain` to view the build output, `--no-cache` to ensure the tests always run, and `--target run-test-stage` to target the test stage. - -```console -$ docker build -f Dockerfile.multistage -t docker-gs-ping-test --progress plain --no-cache --target run-test-stage . -``` - -You should see output containing the following. - -```text -#13 [run-test-stage 1/1] RUN go test -v ./... -#13 4.915 === RUN TestIntMinBasic -#13 4.915 --- PASS: TestIntMinBasic (0.00s) -#13 4.915 === RUN TestIntMinTableDriven -#13 4.915 === RUN TestIntMinTableDriven/0,1 -#13 4.915 === RUN TestIntMinTableDriven/1,0 -#13 4.915 === RUN TestIntMinTableDriven/2,-2 -#13 4.915 === RUN TestIntMinTableDriven/0,-1 -#13 4.915 === RUN TestIntMinTableDriven/-1,0 -#13 4.915 --- PASS: TestIntMinTableDriven (0.00s) -#13 4.915 --- PASS: TestIntMinTableDriven/0,1 (0.00s) -#13 4.915 --- PASS: TestIntMinTableDriven/1,0 (0.00s) -#13 4.915 --- PASS: TestIntMinTableDriven/2,-2 (0.00s) -#13 4.915 --- PASS: TestIntMinTableDriven/0,-1 (0.00s) -#13 4.915 --- PASS: TestIntMinTableDriven/-1,0 (0.00s) -#13 4.915 PASS -``` - -## Next steps - -In this section, you learned how to run tests when building your image. Next, -you’ll learn how to set up a CI/CD pipeline using GitHub Actions. diff --git a/content/guides/grafana-mcp-server-gemini.md b/content/guides/grafana-mcp-server-gemini.md index eed65cfbfc64..c11c32e3c55c 100644 --- a/content/guides/grafana-mcp-server-gemini.md +++ b/content/guides/grafana-mcp-server-gemini.md @@ -5,6 +5,7 @@ title: Connect Gemini to Grafana via MCP summary: | Learn how to leverage the Model Context Protocol (MCP) to interact with Grafana dashboards and datasources directly from your terminal. params: + tags: [ai] proficiencyLevel: Intermediate time: 15 minutes --- diff --git a/content/guides/java/_index.md b/content/guides/java/_index.md index b605ffd48457..96d1da579d21 100644 --- a/content/guides/java/_index.md +++ b/content/guides/java/_index.md @@ -5,16 +5,27 @@ keywords: java, getting started description: Containerize Java apps using Docker summary: | This guide demonstrates how to containerize Java applications using Docker. -toc_min: 1 -toc_max: 2 aliases: - /language/java/ - /guides/language/java/ -languages: [java] + - /language/java/build-images/ + - /language/java/run-containers/ + - /language/java/containerize/ + - /language/java/develop/ + - /language/java/run-tests/ + - /language/java/configure-ci-cd/ + - /language/java/deploy/ + - /guides/java/configure-ci-cd/ + - /guides/java/containerize/ + - /guides/java/deploy/ + - /guides/java/develop/ + - /guides/java/run-tests/ params: + tags: [cicd] time: 20 minutes --- + The Java getting started guide teaches you how to create a containerized Spring Boot application using Docker. In this module, you’ll learn how to: - Containerize and run a Spring Boot application with Maven @@ -26,3 +37,1075 @@ The Java getting started guide teaches you how to create a containerized Spring After completing the Java getting started modules, you should be able to containerize your own Java application based on the examples and instructions provided in this guide. Get started containerizing your first Java app. + +## Containerize a Java application + +### Prerequisites + +- You have installed the latest version of [Docker Desktop](/get-started/get-docker.md). + Docker adds new features regularly and some parts of this guide may + work only with the latest version of Docker Desktop. + +* You have a [Git client](https://git-scm.com/downloads). The examples in this + section use a command-line based Git client, but you can use any client. + +### Overview + +This section walks you through containerizing and running a Java +application. + +### Get the sample applications + +Clone the sample application that you'll be using to your local development machine. Run the following command in a terminal to clone the repository. + +```console +$ git clone https://github.com/spring-projects/spring-petclinic.git +``` + +The sample application is a Spring Boot application built using Maven. For more details, see `readme.md` in the repository. + +### Create Docker assets + +Now that you have an application, you can create the necessary Docker assets to +containerize your application. + +> [!TIP] +> +> [Gordon](/ai/gordon/), Docker's AI assistant, can generate Docker assets for your project. Ask Gordon to create a Dockerfile, Compose file, and `.dockerignore` tailored to your application. + +Create a file named `Dockerfile` with the following contents. + +```dockerfile {collapse=true,title=Dockerfile} +# syntax=docker/dockerfile:1 + +# Comments are provided throughout this file to help you get started. +# If you need more help, visit the Dockerfile reference guide at +# https://docs.docker.com/go/dockerfile-reference/ + +################################################################################ + +# Create a stage for resolving and downloading dependencies. +FROM eclipse-temurin:21-jdk-jammy as deps + +WORKDIR /build + +# Copy the mvnw wrapper with executable permissions. +COPY --chmod=0755 mvnw mvnw +COPY .mvn/ .mvn/ + +# Download dependencies as a separate step to take advantage of Docker's caching. +# Leverage a cache mount to /root/.m2 so that subsequent builds don't have to +# re-download packages. +RUN --mount=type=bind,source=pom.xml,target=pom.xml \ + --mount=type=cache,target=/root/.m2 ./mvnw dependency:go-offline -DskipTests + +################################################################################ + +# Create a stage for building the application based on the stage with downloaded dependencies. +# This Dockerfile is optimized for Java applications that output an uber jar, which includes +# all the dependencies needed to run your app inside a JVM. If your app doesn't output an uber +# jar and instead relies on an application server like Apache Tomcat, you'll need to update this +# stage with the correct filename of your package and update the base image of the "final" stage +# use the relevant app server, e.g., using tomcat (https://hub.docker.com/_/tomcat/) as a base image. +FROM deps as package + +WORKDIR /build + +COPY ./src src/ +RUN --mount=type=bind,source=pom.xml,target=pom.xml \ + --mount=type=cache,target=/root/.m2 \ + ./mvnw package -DskipTests && \ + mv target/$(./mvnw help:evaluate -Dexpression=project.artifactId -q -DforceStdout)-$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout).jar target/app.jar + +################################################################################ + +# Create a stage for extracting the application into separate layers. +# Take advantage of Spring Boot's layer tools and Docker's caching by extracting +# the packaged application into separate layers that can be copied into the final stage. +# See Spring's docs for reference: +# https://docs.spring.io/spring-boot/docs/current/reference/html/container-images.html +FROM package as extract + +WORKDIR /build + +RUN java -Djarmode=layertools -jar target/app.jar extract --destination target/extracted + +################################################################################ + +# Create a new stage for running the application that contains the minimal +# runtime dependencies for the application. This often uses a different base +# image from the install or build stage where the necessary files are copied +# from the install stage. +# +# The example below uses eclipse-turmin's JRE image as the foundation for running the app. +# By specifying the "17-jre-jammy" tag, it will also use whatever happens to be the +# most recent version of that tag when you build your Dockerfile. +# If reproducibility is important, consider using a specific digest SHA, like +# eclipse-temurin@sha256:99cede493dfd88720b610eb8077c8688d3cca50003d76d1d539b0efc8cca72b4. +FROM eclipse-temurin:21-jre-jammy AS final + +# Create a non-privileged user that the app will run under. +# See https://docs.docker.com/go/dockerfile-user-best-practices/ +ARG UID=10001 +RUN adduser \ + --disabled-password \ + --gecos "" \ + --home "/nonexistent" \ + --shell "/sbin/nologin" \ + --no-create-home \ + --uid "${UID}" \ + appuser +USER appuser + +# Copy the executable from the "package" stage. +COPY --from=extract build/target/extracted/dependencies/ ./ +COPY --from=extract build/target/extracted/spring-boot-loader/ ./ +COPY --from=extract build/target/extracted/snapshot-dependencies/ ./ +COPY --from=extract build/target/extracted/application/ ./ + +EXPOSE 8080 + +ENTRYPOINT [ "java", "org.springframework.boot.loader.launch.JarLauncher" ] +``` + +> [!NOTE] +> The sample repository includes a `docker-compose.yml` file. The following instructions use the preferred `compose.yaml` filename — both are supported by Docker Compose. + +Create a file named `compose.yaml` with the following contents. + +```yaml {collapse=true,title=compose.yaml} +# Comments are provided throughout this file to help you get started. +# If you need more help, visit the Docker Compose reference guide at +# https://docs.docker.com/go/compose-spec-reference/ + +# Here the instructions define your application as a service called "server". +# This service is built from the Dockerfile in the current directory. +# You can add other services your application may depend on here, such as a +# database or a cache. For examples, see the Awesome Compose repository: +# https://github.com/docker/awesome-compose +services: + server: + build: + context: . + ports: + - 8080:8080 +# The commented out section below is an example of how to define a PostgreSQL +# database that your application can use. `depends_on` tells Docker Compose to +# start the database before your application. The `db-data` volume persists the +# database data between container restarts. The `db-password` secret is used +# to set the database password. You must create `db/password.txt` and add +# a password of your choosing to it before running `docker compose up`. +# depends_on: +# db: +# condition: service_healthy +# db: +# image: postgres:18 +# restart: always +# user: postgres +# secrets: +# - db-password +# volumes: +# - db-data:/var/lib/postgresql +# environment: +# - POSTGRES_DB=example +# - POSTGRES_PASSWORD_FILE=/run/secrets/db-password +# expose: +# - 5432 +# healthcheck: +# test: [ "CMD", "pg_isready" ] +# interval: 10s +# timeout: 5s +# retries: 5 +# volumes: +# db-data: +# secrets: +# db-password: +# file: db/password.txt + +``` + +Create a file named `.dockerignore` with the following contents. + +```text {collapse=true,title=".dockerignore"} +# Include any files or directories that you don't want to be copied to your +# container here (e.g., local build artifacts, temporary files, etc.). +# +# For more help, visit the .dockerignore file reference guide at +# https://docs.docker.com/go/build-context-dockerignore/ + +**/.classpath +**/.dockerignore +**/.env +**/.git +**/.gitignore +**/.project +**/.settings +**/.toolstarget +**/.vs +**/.vscode +**/.next +**/.cache +**/*.*proj.user +**/*.dbmdl +**/*.jfm +**/charts +**/docker-compose* +**/compose.y*ml +**/target +**/Dockerfile* +**/node_modules +**/npm-debug.log +**/obj +**/secrets.dev.yaml +**/values.dev.yaml +**/vendor +LICENSE +README.md +``` + +You should now have the following three files in your `spring-petclinic` +directory. + +- [Dockerfile](/reference/dockerfile/) +- [.dockerignore](/reference/dockerfile/#dockerignore-file) +- [compose.yaml](/reference/compose-file/_index.md) + +### Run the application + +Inside the `spring-petclinic` directory, run the following command in a +terminal. + +```console +$ docker compose up --build +``` + +The first time you build and run the app, Docker downloads dependencies and builds the app. It may take several minutes depending on your network connection. + +Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see a simple app for a pet clinic. + +In the terminal, press `ctrl`+`c` to stop the application. + +#### Run the application in the background + +You can run the application detached from the terminal by adding the `-d` +option. Inside the `spring-petclinic` directory, run the following command +in a terminal. + +```console +$ docker compose up --build -d +``` + +Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see a simple app for a pet clinic. + +In the terminal, run the following command to stop the application. + +```console +$ docker compose down +``` + +For more information about Compose commands, see the +[Compose CLI reference](/reference/cli/docker/compose/). + +### Summary + +In this section, you learned how you can containerize and run a Java +application using Docker. + +### Next steps + +In the next section, you'll learn how you can develop your application using +Docker containers. + +## Use containers for Java development + +### Prerequisites + +Work through the steps to containerize your application in [Containerize your app](containerize.md). + +### Overview + +In this section, you’ll walk through setting up a local development environment +for the application you containerized in the previous section. This includes: + +- Adding a local database and persisting data +- Creating a development container to connect a debugger +- Configuring Compose to automatically update your running Compose services as + you edit and save your code + +### Add a local database and persist data + +You can use containers to set up local services, like a database. In this section, you'll update the `docker-compose.yaml` file to define a database service and a volume to persist data. Also, this particular application uses a system property to define the database type, so you'll need to update the `Dockerfile` to pass in the system property when starting the app. + +In the cloned repository's directory, open the `docker-compose.yaml` file in an IDE or text editor. Your Compose file has an example database service, but it'll require a few changes for your unique app. + +In the `docker-compose.yaml` file, you need to do the following: + +- Uncomment all of the database instructions. You'll now use a database service + instead of local storage for the data. +- Remove the top-level `secrets` element as well as the element inside the `db` + service. This example uses the environment variable for the password rather than secrets. +- Remove the `user` element from the `db` service. This example specifies the + user in the environment variable. +- Update the database environment variables. These are defined by the Postgres + image. For more details, see the + [Postgres Official Docker Image](https://hub.docker.com/_/postgres). +- Update the healthcheck test for the `db` service and specify the user. By + default, the healthcheck uses the root user instead of the `petclinic` user + you defined. +- Add the database URL as an environment variable in the `server` service. This + overrides the default value defined in + `spring-petclinic/src/main/resources/application-postgres.properties`. + +The following is the updated `docker-compose.yaml` file. All comments have been removed. + +```yaml {hl_lines="7-29"} +services: + server: + build: + context: . + ports: + - 8080:8080 + depends_on: + db: + condition: service_healthy + environment: + - POSTGRES_URL=jdbc:postgresql://db:5432/petclinic + db: + image: postgres:18 + restart: always + volumes: + - db-data:/var/lib/postgresql + environment: + - POSTGRES_DB=petclinic + - POSTGRES_USER=petclinic + - POSTGRES_PASSWORD=petclinic + ports: + - 5432:5432 + healthcheck: + test: ["CMD", "pg_isready", "-U", "petclinic"] + interval: 10s + timeout: 5s + retries: 5 +volumes: + db-data: +``` + +Open the `Dockerfile` in an IDE or text editor. In the `ENTRYPOINT` instruction, +update the instruction to pass in the system property as specified in the +`spring-petclinic/src/resources/db/postgres/petclinic_db_setup_postgres.txt` +file. + +```diff +- ENTRYPOINT [ "java", "org.springframework.boot.loader.launch.JarLauncher" ] ++ ENTRYPOINT [ "java", "-Dspring.profiles.active=postgres", "org.springframework.boot.loader.launch.JarLauncher" ] +``` + +Save and close all the files. + +Now, run the following `docker compose up` command to start your application. + +```console +$ docker compose up --build +``` + +Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see a simple app for a pet clinic. Browse around the application. Navigate to **Veterinarians** and verify that the application is connected to the database by being able to list veterinarians. + +In the terminal, press `ctrl`+`c` to stop the application. + +### Dockerfile for development + +The Dockerfile you have now is great for a small, secure production image with +only the components necessary to run the application. When developing, you may +want a different image that has a different environment. + +For example, in the development image you may want to set up the image to start +the application so that you can connect a debugger to the running Java process. + +Rather than managing multiple Dockerfiles, you can add a new stage. Your +Dockerfile can then produce a final image which is ready for production as well +as a development image. + +Replace the contents of your Dockerfile with the following. + +```dockerfile {hl_lines="22-29"} +# syntax=docker/dockerfile:1 + +FROM eclipse-temurin:21-jdk-jammy as deps +WORKDIR /build +COPY --chmod=0755 mvnw mvnw +COPY .mvn/ .mvn/ +RUN --mount=type=bind,source=pom.xml,target=pom.xml \ + --mount=type=cache,target=/root/.m2 ./mvnw dependency:go-offline -DskipTests + +FROM deps as package +WORKDIR /build +COPY ./src src/ +RUN --mount=type=bind,source=pom.xml,target=pom.xml \ + --mount=type=cache,target=/root/.m2 \ + ./mvnw package -DskipTests && \ + mv target/$(./mvnw help:evaluate -Dexpression=project.artifactId -q -DforceStdout)-$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout).jar target/app.jar + +FROM package as extract +WORKDIR /build +RUN java -Djarmode=layertools -jar target/app.jar extract --destination target/extracted + +FROM extract as development +WORKDIR /build +RUN cp -r /build/target/extracted/dependencies/. ./ +RUN cp -r /build/target/extracted/spring-boot-loader/. ./ +RUN cp -r /build/target/extracted/snapshot-dependencies/. ./ +RUN cp -r /build/target/extracted/application/. ./ +ENV JAVA_TOOL_OPTIONS -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:8000 +CMD [ "java", "-Dspring.profiles.active=postgres", "org.springframework.boot.loader.launch.JarLauncher" ] + +FROM eclipse-temurin:21-jre-jammy AS final +ARG UID=10001 +RUN adduser \ + --disabled-password \ + --gecos "" \ + --home "/nonexistent" \ + --shell "/sbin/nologin" \ + --no-create-home \ + --uid "${UID}" \ + appuser +USER appuser +COPY --from=extract build/target/extracted/dependencies/ ./ +COPY --from=extract build/target/extracted/spring-boot-loader/ ./ +COPY --from=extract build/target/extracted/snapshot-dependencies/ ./ +COPY --from=extract build/target/extracted/application/ ./ +EXPOSE 8080 +ENTRYPOINT [ "java", "-Dspring.profiles.active=postgres", "org.springframework.boot.loader.launch.JarLauncher" ] +``` + +Save and close the `Dockerfile`. + +In the `Dockerfile` you added a new stage labeled `development` based on the `extract` stage. In this stage, you copy the extracted files to a common directory, then run a command to start the application. In the command, you expose port 8000 and declare the debug configuration for the JVM so that you can attach a debugger. + +### Use Compose to develop locally + +The current Compose file doesn't start your development container. To do that, you must update your Compose file to target the development stage. Also, update the port mapping of the server service to provide access for the debugger. + +Open the `docker-compose.yaml` and add the following instructions into the file. + +```yaml {hl_lines=["5","8"]} +services: + server: + build: + context: . + target: development + ports: + - 8080:8080 + - 8000:8000 + depends_on: + db: + condition: service_healthy + environment: + - POSTGRES_URL=jdbc:postgresql://db:5432/petclinic + db: + image: postgres:18 + restart: always + volumes: + - db-data:/var/lib/postgresql + environment: + - POSTGRES_DB=petclinic + - POSTGRES_USER=petclinic + - POSTGRES_PASSWORD=petclinic + ports: + - 5432:5432 + healthcheck: + test: ["CMD", "pg_isready", "-U", "petclinic"] + interval: 10s + timeout: 5s + retries: 5 +volumes: + db-data: +``` + +Now, start your application and to confirm that it's running. + +```console +$ docker compose up --build +``` + +Finally, test your API endpoint. Run the following curl command: + +```console +$ curl --request GET \ + --url http://localhost:8080/vets \ + --header 'content-type: application/json' +``` + +You should receive the following response: + +```json +{ + "vetList": [ + { + "id": 1, + "firstName": "James", + "lastName": "Carter", + "specialties": [], + "nrOfSpecialties": 0, + "new": false + }, + { + "id": 2, + "firstName": "Helen", + "lastName": "Leary", + "specialties": [{ "id": 1, "name": "radiology", "new": false }], + "nrOfSpecialties": 1, + "new": false + }, + { + "id": 3, + "firstName": "Linda", + "lastName": "Douglas", + "specialties": [ + { "id": 3, "name": "dentistry", "new": false }, + { "id": 2, "name": "surgery", "new": false } + ], + "nrOfSpecialties": 2, + "new": false + }, + { + "id": 4, + "firstName": "Rafael", + "lastName": "Ortega", + "specialties": [{ "id": 2, "name": "surgery", "new": false }], + "nrOfSpecialties": 1, + "new": false + }, + { + "id": 5, + "firstName": "Henry", + "lastName": "Stevens", + "specialties": [{ "id": 1, "name": "radiology", "new": false }], + "nrOfSpecialties": 1, + "new": false + }, + { + "id": 6, + "firstName": "Sharon", + "lastName": "Jenkins", + "specialties": [], + "nrOfSpecialties": 0, + "new": false + } + ] +} +``` + +### Connect a Debugger + +You’ll use the debugger that comes with the IntelliJ IDEA. You can use the community version of this IDE. Open your project in IntelliJ IDEA, go to the **Run** menu, and then **Edit Configuration**. Add a new Remote JVM Debug configuration similar to the following: + +![Java Connect a Debugger](images/connect-debugger.webp) + +Set a breakpoint. + +Open `src/main/java/org/springframework/samples/petclinic/vet/VetController.java` and add a breakpoint inside the `showResourcesVetList` function. + +To start your debug session, select the **Run** menu and then **Debug _NameOfYourConfiguration_**. + +![Debug menu](images/debug-menu.webp?w=300) + +You should now see the connection in the logs of your Compose application. + +![Compose log file ](images/compose-logs.webp) + +You can now call the server endpoint. + +```console +$ curl --request GET --url http://localhost:8080/vets +``` + +You should have seen the code break on the marked line and now you are able to use the debugger just like you would normally. You can also inspect and watch variables, set conditional breakpoints, view stack traces and a do bunch of other stuff. + +![Debugger code breakpoint](images/debugger-breakpoint.webp) + +Press `ctrl+c` in the terminal to stop your application. + +### Automatically update services + +Use Compose Watch to automatically update your running Compose services as you +edit and save your code. For more details about Compose Watch, see +[Use Compose Watch](/manuals/compose/how-tos/file-watch.md). + +Open your `docker-compose.yaml` file in an IDE or text editor and then add the +Compose Watch instructions. The following is the updated `docker-compose.yaml` +file. + +```yaml {hl_lines="14-17"} +services: + server: + build: + context: . + target: development + ports: + - 8080:8080 + - 8000:8000 + depends_on: + db: + condition: service_healthy + environment: + - POSTGRES_URL=jdbc:postgresql://db:5432/petclinic + develop: + watch: + - action: rebuild + path: . + db: + image: postgres:18 + restart: always + volumes: + - db-data:/var/lib/postgresql + environment: + - POSTGRES_DB=petclinic + - POSTGRES_USER=petclinic + - POSTGRES_PASSWORD=petclinic + ports: + - 5432:5432 + healthcheck: + test: ["CMD", "pg_isready", "-U", "petclinic"] + interval: 10s + timeout: 5s + retries: 5 +volumes: + db-data: +``` + +Run the following command to run your application with Compose Watch. + +```console +$ docker compose watch +``` + +Open a web browser and view the application at [http://localhost:8080](http://localhost:8080). You should see the Spring Pet Clinic home page. + +Any changes to the application's source files on your local machine will now be automatically reflected in the running container. + +Open `spring-petclinic/src/main/resources/templates/fragments/layout.html` in an IDE or text editor and update the `Home` navigation string by adding an exclamation mark. + +```diff +-
  • ++
  • + +``` + +Save the changes to `layout.html` and then you can continue developing while the container automatically rebuilds. + +After the container is rebuilt and running, refresh [http://localhost:8080](http://localhost:8080) and then verify that **Home!** now appears in the menu. + +Press `ctrl+c` in the terminal to stop Compose Watch. + +### Summary + +In this section, you took a look at running a database locally and persisting the data. You also created a development image that contains the JDK and lets you attach a debugger. Finally, you set up your Compose file to expose the debugging port and configured Compose Watch to live reload your changes. + +Related information: + +- [Compose file reference](/reference/compose-file/) +- [Compose Watch](/manuals/compose/how-tos/file-watch.md) +- [Dockerfile reference](/reference/dockerfile/) + +### Next steps + +In the next section, you’ll take a look at how to run unit tests in Docker. + +## Run your Java tests + +### Prerequisites + +Complete all the previous sections of this guide, starting with [Containerize a Java application](containerize.md). + +### Overview + +Testing is an essential part of modern software development. Testing can mean a lot of things to different development teams. There are unit tests, integration tests and end-to-end testing. In this guide you'll take a look at running your unit tests in Docker. + +#### Multi-stage Dockerfile for testing + +In the following example, you'll pull the testing commands into your Dockerfile. +Replace the contents of your Dockerfile with the following. + +```dockerfile {hl_lines="3-19"} +# syntax=docker/dockerfile:1 + +FROM eclipse-temurin:21-jdk-jammy as base +WORKDIR /build +COPY --chmod=0755 mvnw mvnw +COPY .mvn/ .mvn/ + +FROM base as test +WORKDIR /build +COPY ./src src/ +RUN --mount=type=bind,source=pom.xml,target=pom.xml \ + --mount=type=cache,target=/root/.m2 \ + ./mvnw test + +FROM base as deps +WORKDIR /build +RUN --mount=type=bind,source=pom.xml,target=pom.xml \ + --mount=type=cache,target=/root/.m2 \ + ./mvnw dependency:go-offline -DskipTests + +FROM deps as package +WORKDIR /build +COPY ./src src/ +RUN --mount=type=bind,source=pom.xml,target=pom.xml \ + --mount=type=cache,target=/root/.m2 \ + ./mvnw package -DskipTests && \ + mv target/$(./mvnw help:evaluate -Dexpression=project.artifactId -q -DforceStdout)-$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout).jar target/app.jar + +FROM package as extract +WORKDIR /build +RUN java -Djarmode=layertools -jar target/app.jar extract --destination target/extracted + +FROM extract as development +WORKDIR /build +RUN cp -r /build/target/extracted/dependencies/. ./ +RUN cp -r /build/target/extracted/spring-boot-loader/. ./ +RUN cp -r /build/target/extracted/snapshot-dependencies/. ./ +RUN cp -r /build/target/extracted/application/. ./ +ENV JAVA_TOOL_OPTIONS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:8000" +CMD [ "java", "-Dspring.profiles.active=postgres", "org.springframework.boot.loader.launch.JarLauncher" ] + +FROM eclipse-temurin:21-jre-jammy AS final +ARG UID=10001 +RUN adduser \ + --disabled-password \ + --gecos "" \ + --home "/nonexistent" \ + --shell "/sbin/nologin" \ + --no-create-home \ + --uid "${UID}" \ + appuser +USER appuser +COPY --from=extract build/target/extracted/dependencies/ ./ +COPY --from=extract build/target/extracted/spring-boot-loader/ ./ +COPY --from=extract build/target/extracted/snapshot-dependencies/ ./ +COPY --from=extract build/target/extracted/application/ ./ +EXPOSE 8080 +ENTRYPOINT [ "java", "-Dspring.profiles.active=postgres", "org.springframework.boot.loader.launch.JarLauncher" ] +``` + +First, you added a new base stage. In the base stage, you added common instructions that both the test and deps stage will need. + +Next, you added a new test stage labeled `test` based on the base stage. In this +stage you copied in the necessary source files and then specified `RUN` to run +`./mvnw test`. Instead of using `CMD`, you used `RUN` to run the tests. The +reason is that the `CMD` instruction runs when the container runs, and the `RUN` +instruction runs when the image is being built. When using `RUN`, the build will +fail if the tests fail. + +Finally, you updated the deps stage to be based on the base stage and removed +the instructions that are now in the base stage. + +Run the following command to build a new image using the test stage as the target and view the test results. Include `--progress=plain` to view the build output, `--no-cache` to ensure the tests always run, and `--target test` to target the test stage. + +Now, build your image and run your tests. You'll run the `docker build` command and add the `--target test` flag so that you specifically run the test build stage. + +```console +$ docker build -t java-docker-image-test --progress=plain --no-cache --target=test . +``` + +You should see output containing the following + +```console +... + +#15 101.3 [WARNING] Tests run: 45, Failures: 0, Errors: 0, Skipped: 2 +#15 101.3 [INFO] +#15 101.3 [INFO] ------------------------------------------------------------------------ +#15 101.3 [INFO] BUILD SUCCESS +#15 101.3 [INFO] ------------------------------------------------------------------------ +#15 101.3 [INFO] Total time: 01:39 min +#15 101.3 [INFO] Finished at: 2024-02-01T23:24:48Z +#15 101.3 [INFO] ------------------------------------------------------------------------ +#15 DONE 101.4s +``` + +### Next steps + +In the next section, you’ll take a look at how to set up a CI/CD pipeline using +GitHub Actions. + +## Configure CI/CD for your Java application + +### Prerequisites + +Complete the previous sections of this guide, starting with [Containerize your app](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. + +### Overview + +In this section, you'll learn how to set up and use GitHub Actions to build and push your Docker image to Docker Hub. You will complete the following steps: + +1. Create a new repository on GitHub. +2. Define the GitHub Actions workflow. +3. Run the workflow. + +### Step one: Create the repository + +Create a GitHub repository, configure the Docker Hub credentials, and push your source code. + +1. [Create a new repository](https://github.com/new) on GitHub. + +2. Open the repository **Settings**, and go to **Secrets and variables** > + **Actions**. + +3. Create a new **Repository variable** named `DOCKER_USERNAME` and your Docker ID as a value. + +4. Create a new [Personal Access Token (PAT)](/manuals/security/access-tokens.md#create-an-access-token) for Docker Hub. You can name this token `docker-tutorial`. Make sure access permissions include Read and Write. + +5. Add the PAT as a **Repository secret** in your GitHub repository, with the name + `DOCKERHUB_TOKEN`. + +6. In your local repository on your machine, run the following command to change + the origin to the repository you just created. Make sure you change + `your-username` to your GitHub username and `your-repository` to the name of + the repository you created. + + ```console + $ git remote set-url origin https://github.com/your-username/your-repository.git + ``` + +7. Run the following commands to stage, commit, and push your local repository to GitHub. + + ```console + $ git add -A + $ git commit -m "my commit" + $ git push -u origin main + ``` + +### Step two: Set up the workflow + +Set up your GitHub Actions workflow for building, testing, and pushing the image +to Docker Hub. + +1. Go to your repository on GitHub and then select the **Actions** tab. + The project already has the `maven-build` workflow to build and test your Java application with Maven. If you want, you can optionally disable this workflow because you won't use it in this guide. You'll create a new, alternate workflow to build, test, and push your image. + +2. Select **New workflow**. + +3. Select **set up a workflow yourself**. + + This takes you to a page for creating a new GitHub actions workflow file in + your repository, under `.github/workflows/main.yml` by default. + +4. In the editor window, copy and paste the following YAML configuration. + + ```yaml + name: ci + + on: + push: + branches: + - main + + jobs: + build: + runs-on: ubuntu-latest + steps: + - name: Login to Docker Hub + uses: docker/login-action@{{% param "login_action_version" %}} + with: + username: ${{ vars.DOCKER_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} + + - name: Build and test + uses: docker/build-push-action@{{% param "build_push_action_version" %}} + with: + target: test + load: true + + - name: Build and push + uses: docker/build-push-action@{{% param "build_push_action_version" %}} + with: + platforms: linux/amd64,linux/arm64 + push: true + target: final + tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest + ``` + + For more information about the YAML syntax for `docker/build-push-action`, + refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). + +### Step three: Run the workflow + +Save the workflow file and run the job. + +1. Select **Commit changes...** and push the changes to the `main` branch. + + After pushing the commit, the workflow starts automatically. + +2. Go to the **Actions** tab. It displays the workflow. + + Selecting the workflow shows you the breakdown of all the steps. + +3. When the workflow is complete, go to your + [repositories on Docker Hub](https://hub.docker.com/repositories). + + If you see the new repository in that list, it means the GitHub Actions + successfully pushed the image to Docker Hub. + +### Summary + +In this section, you learned how to set up a GitHub Actions workflow for your application. + +Related information: + +- [Introduction to GitHub Actions](/guides/gha.md) +- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) +- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) + +### Next steps + +Next, learn how you can locally test and debug your workloads on Kubernetes before deploying. + +## Test your Java deployment + +### Prerequisites + +- Complete all the previous sections of this guide, starting with [Containerize your app](containerize.md). +- [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. + +### Overview + +In this section, you'll learn how to use Docker Desktop to deploy your +application to a fully-featured Kubernetes environment on your development +machine. This lets you test and debug your workloads on Kubernetes locally +before deploying. + +### Create a Kubernetes YAML file + +In your `spring-petclinic` directory, create a file named +`docker-java-kubernetes.yaml`. Open the file in an IDE or text editor and add +the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker +username and the name of the repository that you created in [Configure CI/CD for +your Java application](configure-ci-cd.md). + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: docker-java-demo + namespace: default +spec: + replicas: 1 + selector: + matchLabels: + service: server + template: + metadata: + labels: + service: server + spec: + containers: + - name: server-service + image: DOCKER_USERNAME/REPO_NAME + imagePullPolicy: Always +--- +apiVersion: v1 +kind: Service +metadata: + name: service-entrypoint + namespace: default +spec: + type: NodePort + selector: + service: server + ports: + - port: 8080 + targetPort: 8080 + nodePort: 30001 +``` + +In this Kubernetes YAML file, there are two objects, separated by the `---`: + +- A Deployment, describing a scalable group of identical pods. In this case, + you'll get just one replica, or copy of your pod. That pod, which is + described under `template`, has just one container in it. The + container is created from the image built by GitHub Actions in [Configure CI/CD for + your Java application](configure-ci-cd.md). +- A NodePort service, which will route traffic from port 30001 on your host to + port 8080 inside the pods it routes to, allowing you to reach your app + from the network. + +To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). + +### Deploy and check your application + +1. In a terminal, navigate to `spring-petclinic` and deploy your application to + Kubernetes. + + ```console + $ kubectl apply -f docker-java-kubernetes.yaml + ``` + + You should see output that looks like the following, indicating your Kubernetes objects were created successfully. + + ```shell + deployment.apps/docker-java-demo created + service/service-entrypoint created + ``` + +2. Make sure everything worked by listing your deployments. + + ```console + $ kubectl get deployments + ``` + + Your deployment should be listed as follows: + + ```shell + NAME READY UP-TO-DATE AVAILABLE AGE + docker-java-demo 1/1 1 1 15s + ``` + + This indicates all one of the pods you asked for in your YAML are up and running. Do the same check for your services. + + ```console + $ kubectl get services + ``` + + You should get output like the following. + + ```shell + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + kubernetes ClusterIP 10.96.0.1 443/TCP 23h + service-entrypoint NodePort 10.99.128.230 8080:30001/TCP 75s + ``` + + In addition to the default `kubernetes` service, you can see your `service-entrypoint` service, accepting traffic on port 30001/TCP. + +3. In a terminal, curl the service. Note that a database wasn't deployed in + this example. + + ```console + $ curl --request GET \ + --url http://localhost:30001/actuator/health \ + --header 'content-type: application/json' + ``` + + You should get output like the following. + + ```console + {"status":"UP","groups":["liveness","readiness"]} + ``` + +4. Run the following command to tear down your application. + + ```console + $ kubectl delete -f docker-java-kubernetes.yaml + ``` + +### Summary + +In this section, you learned how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. + +Related information: + +- [Kubernetes documentation](https://kubernetes.io/docs/home/) +- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) +- [Swarm mode overview](/manuals/engine/swarm/_index.md) diff --git a/content/guides/java/configure-ci-cd.md b/content/guides/java/configure-ci-cd.md deleted file mode 100644 index ee6c0cd41bf7..000000000000 --- a/content/guides/java/configure-ci-cd.md +++ /dev/null @@ -1,143 +0,0 @@ ---- -title: Configure CI/CD for your Java application -linkTitle: Configure CI/CD -weight: 40 -keywords: java, CI/CD, local, development -description: Learn how to Configure CI/CD for your Java application -aliases: - - /language/java/configure-ci-cd/ - - /guides/language/java/configure-ci-cd/ ---- - -## Prerequisites - -Complete the previous sections of this guide, starting with [Containerize your app](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. - -## Overview - -In this section, you'll learn how to set up and use GitHub Actions to build and push your Docker image to Docker Hub. You will complete the following steps: - -1. Create a new repository on GitHub. -2. Define the GitHub Actions workflow. -3. Run the workflow. - -## Step one: Create the repository - -Create a GitHub repository, configure the Docker Hub credentials, and push your source code. - -1. [Create a new repository](https://github.com/new) on GitHub. - -2. Open the repository **Settings**, and go to **Secrets and variables** > - **Actions**. - -3. Create a new **Repository variable** named `DOCKER_USERNAME` and your Docker ID as a value. - -4. Create a new [Personal Access Token (PAT)](/manuals/security/access-tokens.md#create-an-access-token) for Docker Hub. You can name this token `docker-tutorial`. Make sure access permissions include Read and Write. - -5. Add the PAT as a **Repository secret** in your GitHub repository, with the name - `DOCKERHUB_TOKEN`. - -6. In your local repository on your machine, run the following command to change - the origin to the repository you just created. Make sure you change - `your-username` to your GitHub username and `your-repository` to the name of - the repository you created. - - ```console - $ git remote set-url origin https://github.com/your-username/your-repository.git - ``` - -7. Run the following commands to stage, commit, and push your local repository to GitHub. - - ```console - $ git add -A - $ git commit -m "my commit" - $ git push -u origin main - ``` - -## Step two: Set up the workflow - -Set up your GitHub Actions workflow for building, testing, and pushing the image -to Docker Hub. - -1. Go to your repository on GitHub and then select the **Actions** tab. - The project already has the `maven-build` workflow to build and test your Java application with Maven. If you want, you can optionally disable this workflow because you won't use it in this guide. You'll create a new, alternate workflow to build, test, and push your image. - -2. Select **New workflow**. - -3. Select **set up a workflow yourself**. - - This takes you to a page for creating a new GitHub actions workflow file in - your repository, under `.github/workflows/main.yml` by default. - -4. In the editor window, copy and paste the following YAML configuration. - - ```yaml - name: ci - - on: - push: - branches: - - main - - jobs: - build: - runs-on: ubuntu-latest - steps: - - name: Login to Docker Hub - uses: docker/login-action@{{% param "login_action_version" %}} - with: - username: ${{ vars.DOCKER_USERNAME }} - password: ${{ secrets.DOCKERHUB_TOKEN }} - - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} - - - name: Build and test - uses: docker/build-push-action@{{% param "build_push_action_version" %}} - with: - target: test - load: true - - - name: Build and push - uses: docker/build-push-action@{{% param "build_push_action_version" %}} - with: - platforms: linux/amd64,linux/arm64 - push: true - target: final - tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest - ``` - - For more information about the YAML syntax for `docker/build-push-action`, - refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). - -## Step three: Run the workflow - -Save the workflow file and run the job. - -1. Select **Commit changes...** and push the changes to the `main` branch. - - After pushing the commit, the workflow starts automatically. - -2. Go to the **Actions** tab. It displays the workflow. - - Selecting the workflow shows you the breakdown of all the steps. - -3. When the workflow is complete, go to your - [repositories on Docker Hub](https://hub.docker.com/repositories). - - If you see the new repository in that list, it means the GitHub Actions - successfully pushed the image to Docker Hub. - -## Summary - -In this section, you learned how to set up a GitHub Actions workflow for your application. - -Related information: - -- [Introduction to GitHub Actions](/guides/gha.md) -- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) -- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) - -## Next steps - -Next, learn how you can locally test and debug your workloads on Kubernetes before deploying. diff --git a/content/guides/java/containerize.md b/content/guides/java/containerize.md deleted file mode 100644 index 83f6fd4e938c..000000000000 --- a/content/guides/java/containerize.md +++ /dev/null @@ -1,288 +0,0 @@ ---- -title: Containerize a Java application -linkTitle: Containerize your app -weight: 10 -keywords: java, containerize, initialize, maven, build -description: Learn how to containerize a Java application. -aliases: - - /language/java/build-images/ - - /language/java/run-containers/ - - /language/java/containerize/ - - /guides/language/java/containerize/ ---- - -## Prerequisites - -- You have installed the latest version of [Docker Desktop](/get-started/get-docker.md). - Docker adds new features regularly and some parts of this guide may - work only with the latest version of Docker Desktop. - -* You have a [Git client](https://git-scm.com/downloads). The examples in this - section use a command-line based Git client, but you can use any client. - -## Overview - -This section walks you through containerizing and running a Java -application. - -## Get the sample applications - -Clone the sample application that you'll be using to your local development machine. Run the following command in a terminal to clone the repository. - -```console -$ git clone https://github.com/spring-projects/spring-petclinic.git -``` - -The sample application is a Spring Boot application built using Maven. For more details, see `readme.md` in the repository. - -## Create Docker assets - -Now that you have an application, you can create the necessary Docker assets to -containerize your application. - -> [!TIP] -> -> [Gordon](/ai/gordon/), Docker's AI assistant, can generate Docker assets for your project. Ask Gordon to create a Dockerfile, Compose file, and `.dockerignore` tailored to your application. - -Create a file named `Dockerfile` with the following contents. - -```dockerfile {collapse=true,title=Dockerfile} -# syntax=docker/dockerfile:1 - -# Comments are provided throughout this file to help you get started. -# If you need more help, visit the Dockerfile reference guide at -# https://docs.docker.com/go/dockerfile-reference/ - -################################################################################ - -# Create a stage for resolving and downloading dependencies. -FROM eclipse-temurin:21-jdk-jammy as deps - -WORKDIR /build - -# Copy the mvnw wrapper with executable permissions. -COPY --chmod=0755 mvnw mvnw -COPY .mvn/ .mvn/ - -# Download dependencies as a separate step to take advantage of Docker's caching. -# Leverage a cache mount to /root/.m2 so that subsequent builds don't have to -# re-download packages. -RUN --mount=type=bind,source=pom.xml,target=pom.xml \ - --mount=type=cache,target=/root/.m2 ./mvnw dependency:go-offline -DskipTests - -################################################################################ - -# Create a stage for building the application based on the stage with downloaded dependencies. -# This Dockerfile is optimized for Java applications that output an uber jar, which includes -# all the dependencies needed to run your app inside a JVM. If your app doesn't output an uber -# jar and instead relies on an application server like Apache Tomcat, you'll need to update this -# stage with the correct filename of your package and update the base image of the "final" stage -# use the relevant app server, e.g., using tomcat (https://hub.docker.com/_/tomcat/) as a base image. -FROM deps as package - -WORKDIR /build - -COPY ./src src/ -RUN --mount=type=bind,source=pom.xml,target=pom.xml \ - --mount=type=cache,target=/root/.m2 \ - ./mvnw package -DskipTests && \ - mv target/$(./mvnw help:evaluate -Dexpression=project.artifactId -q -DforceStdout)-$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout).jar target/app.jar - -################################################################################ - -# Create a stage for extracting the application into separate layers. -# Take advantage of Spring Boot's layer tools and Docker's caching by extracting -# the packaged application into separate layers that can be copied into the final stage. -# See Spring's docs for reference: -# https://docs.spring.io/spring-boot/docs/current/reference/html/container-images.html -FROM package as extract - -WORKDIR /build - -RUN java -Djarmode=layertools -jar target/app.jar extract --destination target/extracted - -################################################################################ - -# Create a new stage for running the application that contains the minimal -# runtime dependencies for the application. This often uses a different base -# image from the install or build stage where the necessary files are copied -# from the install stage. -# -# The example below uses eclipse-turmin's JRE image as the foundation for running the app. -# By specifying the "17-jre-jammy" tag, it will also use whatever happens to be the -# most recent version of that tag when you build your Dockerfile. -# If reproducibility is important, consider using a specific digest SHA, like -# eclipse-temurin@sha256:99cede493dfd88720b610eb8077c8688d3cca50003d76d1d539b0efc8cca72b4. -FROM eclipse-temurin:21-jre-jammy AS final - -# Create a non-privileged user that the app will run under. -# See https://docs.docker.com/go/dockerfile-user-best-practices/ -ARG UID=10001 -RUN adduser \ - --disabled-password \ - --gecos "" \ - --home "/nonexistent" \ - --shell "/sbin/nologin" \ - --no-create-home \ - --uid "${UID}" \ - appuser -USER appuser - -# Copy the executable from the "package" stage. -COPY --from=extract build/target/extracted/dependencies/ ./ -COPY --from=extract build/target/extracted/spring-boot-loader/ ./ -COPY --from=extract build/target/extracted/snapshot-dependencies/ ./ -COPY --from=extract build/target/extracted/application/ ./ - -EXPOSE 8080 - -ENTRYPOINT [ "java", "org.springframework.boot.loader.launch.JarLauncher" ] -``` - -> [!NOTE] -> The sample repository includes a `docker-compose.yml` file. The following instructions use the preferred `compose.yaml` filename — both are supported by Docker Compose. - -Create a file named `compose.yaml` with the following contents. - -```yaml {collapse=true,title=compose.yaml} -# Comments are provided throughout this file to help you get started. -# If you need more help, visit the Docker Compose reference guide at -# https://docs.docker.com/go/compose-spec-reference/ - -# Here the instructions define your application as a service called "server". -# This service is built from the Dockerfile in the current directory. -# You can add other services your application may depend on here, such as a -# database or a cache. For examples, see the Awesome Compose repository: -# https://github.com/docker/awesome-compose -services: - server: - build: - context: . - ports: - - 8080:8080 -# The commented out section below is an example of how to define a PostgreSQL -# database that your application can use. `depends_on` tells Docker Compose to -# start the database before your application. The `db-data` volume persists the -# database data between container restarts. The `db-password` secret is used -# to set the database password. You must create `db/password.txt` and add -# a password of your choosing to it before running `docker compose up`. -# depends_on: -# db: -# condition: service_healthy -# db: -# image: postgres:18 -# restart: always -# user: postgres -# secrets: -# - db-password -# volumes: -# - db-data:/var/lib/postgresql -# environment: -# - POSTGRES_DB=example -# - POSTGRES_PASSWORD_FILE=/run/secrets/db-password -# expose: -# - 5432 -# healthcheck: -# test: [ "CMD", "pg_isready" ] -# interval: 10s -# timeout: 5s -# retries: 5 -# volumes: -# db-data: -# secrets: -# db-password: -# file: db/password.txt - -``` - -Create a file named `.dockerignore` with the following contents. - -```text {collapse=true,title=".dockerignore"} -# Include any files or directories that you don't want to be copied to your -# container here (e.g., local build artifacts, temporary files, etc.). -# -# For more help, visit the .dockerignore file reference guide at -# https://docs.docker.com/go/build-context-dockerignore/ - -**/.classpath -**/.dockerignore -**/.env -**/.git -**/.gitignore -**/.project -**/.settings -**/.toolstarget -**/.vs -**/.vscode -**/.next -**/.cache -**/*.*proj.user -**/*.dbmdl -**/*.jfm -**/charts -**/docker-compose* -**/compose.y*ml -**/target -**/Dockerfile* -**/node_modules -**/npm-debug.log -**/obj -**/secrets.dev.yaml -**/values.dev.yaml -**/vendor -LICENSE -README.md -``` - -You should now have the following three files in your `spring-petclinic` -directory. - -- [Dockerfile](/reference/dockerfile/) -- [.dockerignore](/reference/dockerfile/#dockerignore-file) -- [compose.yaml](/reference/compose-file/_index.md) - -## Run the application - -Inside the `spring-petclinic` directory, run the following command in a -terminal. - -```console -$ docker compose up --build -``` - -The first time you build and run the app, Docker downloads dependencies and builds the app. It may take several minutes depending on your network connection. - -Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see a simple app for a pet clinic. - -In the terminal, press `ctrl`+`c` to stop the application. - -### Run the application in the background - -You can run the application detached from the terminal by adding the `-d` -option. Inside the `spring-petclinic` directory, run the following command -in a terminal. - -```console -$ docker compose up --build -d -``` - -Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see a simple app for a pet clinic. - -In the terminal, run the following command to stop the application. - -```console -$ docker compose down -``` - -For more information about Compose commands, see the -[Compose CLI reference](/reference/cli/docker/compose/). - -## Summary - -In this section, you learned how you can containerize and run a Java -application using Docker. - -## Next steps - -In the next section, you'll learn how you can develop your application using -Docker containers. diff --git a/content/guides/java/deploy.md b/content/guides/java/deploy.md deleted file mode 100644 index 663fe4fd71ca..000000000000 --- a/content/guides/java/deploy.md +++ /dev/null @@ -1,155 +0,0 @@ ---- -title: Test your Java deployment -linkTitle: Test your deployment -weight: 50 -keywords: deploy, kubernetes, java -description: Learn how to develop locally using Kubernetes -aliases: - - /language/java/deploy/ - - /guides/language/java/deploy/ ---- - -## Prerequisites - -- Complete all the previous sections of this guide, starting with [Containerize your app](containerize.md). -- [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. - -## Overview - -In this section, you'll learn how to use Docker Desktop to deploy your -application to a fully-featured Kubernetes environment on your development -machine. This lets you test and debug your workloads on Kubernetes locally -before deploying. - -## Create a Kubernetes YAML file - -In your `spring-petclinic` directory, create a file named -`docker-java-kubernetes.yaml`. Open the file in an IDE or text editor and add -the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker -username and the name of the repository that you created in [Configure CI/CD for -your Java application](configure-ci-cd.md). - -```yaml -apiVersion: apps/v1 -kind: Deployment -metadata: - name: docker-java-demo - namespace: default -spec: - replicas: 1 - selector: - matchLabels: - service: server - template: - metadata: - labels: - service: server - spec: - containers: - - name: server-service - image: DOCKER_USERNAME/REPO_NAME - imagePullPolicy: Always ---- -apiVersion: v1 -kind: Service -metadata: - name: service-entrypoint - namespace: default -spec: - type: NodePort - selector: - service: server - ports: - - port: 8080 - targetPort: 8080 - nodePort: 30001 -``` - -In this Kubernetes YAML file, there are two objects, separated by the `---`: - -- A Deployment, describing a scalable group of identical pods. In this case, - you'll get just one replica, or copy of your pod. That pod, which is - described under `template`, has just one container in it. The - container is created from the image built by GitHub Actions in [Configure CI/CD for - your Java application](configure-ci-cd.md). -- A NodePort service, which will route traffic from port 30001 on your host to - port 8080 inside the pods it routes to, allowing you to reach your app - from the network. - -To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). - -## Deploy and check your application - -1. In a terminal, navigate to `spring-petclinic` and deploy your application to - Kubernetes. - - ```console - $ kubectl apply -f docker-java-kubernetes.yaml - ``` - - You should see output that looks like the following, indicating your Kubernetes objects were created successfully. - - ```shell - deployment.apps/docker-java-demo created - service/service-entrypoint created - ``` - -2. Make sure everything worked by listing your deployments. - - ```console - $ kubectl get deployments - ``` - - Your deployment should be listed as follows: - - ```shell - NAME READY UP-TO-DATE AVAILABLE AGE - docker-java-demo 1/1 1 1 15s - ``` - - This indicates all one of the pods you asked for in your YAML are up and running. Do the same check for your services. - - ```console - $ kubectl get services - ``` - - You should get output like the following. - - ```shell - NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE - kubernetes ClusterIP 10.96.0.1 443/TCP 23h - service-entrypoint NodePort 10.99.128.230 8080:30001/TCP 75s - ``` - - In addition to the default `kubernetes` service, you can see your `service-entrypoint` service, accepting traffic on port 30001/TCP. - -3. In a terminal, curl the service. Note that a database wasn't deployed in - this example. - - ```console - $ curl --request GET \ - --url http://localhost:30001/actuator/health \ - --header 'content-type: application/json' - ``` - - You should get output like the following. - - ```console - {"status":"UP","groups":["liveness","readiness"]} - ``` - -4. Run the following command to tear down your application. - - ```console - $ kubectl delete -f docker-java-kubernetes.yaml - ``` - -## Summary - -In this section, you learned how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. - -Related information: - -- [Kubernetes documentation](https://kubernetes.io/docs/home/) -- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) -- [Swarm mode overview](/manuals/engine/swarm/_index.md) diff --git a/content/guides/java/develop.md b/content/guides/java/develop.md deleted file mode 100644 index fd4a62a19b03..000000000000 --- a/content/guides/java/develop.md +++ /dev/null @@ -1,403 +0,0 @@ ---- -title: Use containers for Java development -linkTitle: Develop your app -weight: 20 -keywords: Java, local, development, run, -description: Learn how to develop your application locally. -aliases: - - /language/java/develop/ - - /guides/language/java/develop/ ---- - -## Prerequisites - -Work through the steps to containerize your application in [Containerize your app](containerize.md). - -## Overview - -In this section, you’ll walk through setting up a local development environment -for the application you containerized in the previous section. This includes: - -- Adding a local database and persisting data -- Creating a development container to connect a debugger -- Configuring Compose to automatically update your running Compose services as - you edit and save your code - -## Add a local database and persist data - -You can use containers to set up local services, like a database. In this section, you'll update the `docker-compose.yaml` file to define a database service and a volume to persist data. Also, this particular application uses a system property to define the database type, so you'll need to update the `Dockerfile` to pass in the system property when starting the app. - -In the cloned repository's directory, open the `docker-compose.yaml` file in an IDE or text editor. Your Compose file has an example database service, but it'll require a few changes for your unique app. - -In the `docker-compose.yaml` file, you need to do the following: - -- Uncomment all of the database instructions. You'll now use a database service - instead of local storage for the data. -- Remove the top-level `secrets` element as well as the element inside the `db` - service. This example uses the environment variable for the password rather than secrets. -- Remove the `user` element from the `db` service. This example specifies the - user in the environment variable. -- Update the database environment variables. These are defined by the Postgres - image. For more details, see the - [Postgres Official Docker Image](https://hub.docker.com/_/postgres). -- Update the healthcheck test for the `db` service and specify the user. By - default, the healthcheck uses the root user instead of the `petclinic` user - you defined. -- Add the database URL as an environment variable in the `server` service. This - overrides the default value defined in - `spring-petclinic/src/main/resources/application-postgres.properties`. - -The following is the updated `docker-compose.yaml` file. All comments have been removed. - -```yaml {hl_lines="7-29"} -services: - server: - build: - context: . - ports: - - 8080:8080 - depends_on: - db: - condition: service_healthy - environment: - - POSTGRES_URL=jdbc:postgresql://db:5432/petclinic - db: - image: postgres:18 - restart: always - volumes: - - db-data:/var/lib/postgresql - environment: - - POSTGRES_DB=petclinic - - POSTGRES_USER=petclinic - - POSTGRES_PASSWORD=petclinic - ports: - - 5432:5432 - healthcheck: - test: ["CMD", "pg_isready", "-U", "petclinic"] - interval: 10s - timeout: 5s - retries: 5 -volumes: - db-data: -``` - -Open the `Dockerfile` in an IDE or text editor. In the `ENTRYPOINT` instruction, -update the instruction to pass in the system property as specified in the -`spring-petclinic/src/resources/db/postgres/petclinic_db_setup_postgres.txt` -file. - -```diff -- ENTRYPOINT [ "java", "org.springframework.boot.loader.launch.JarLauncher" ] -+ ENTRYPOINT [ "java", "-Dspring.profiles.active=postgres", "org.springframework.boot.loader.launch.JarLauncher" ] -``` - -Save and close all the files. - -Now, run the following `docker compose up` command to start your application. - -```console -$ docker compose up --build -``` - -Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see a simple app for a pet clinic. Browse around the application. Navigate to **Veterinarians** and verify that the application is connected to the database by being able to list veterinarians. - -In the terminal, press `ctrl`+`c` to stop the application. - -## Dockerfile for development - -The Dockerfile you have now is great for a small, secure production image with -only the components necessary to run the application. When developing, you may -want a different image that has a different environment. - -For example, in the development image you may want to set up the image to start -the application so that you can connect a debugger to the running Java process. - -Rather than managing multiple Dockerfiles, you can add a new stage. Your -Dockerfile can then produce a final image which is ready for production as well -as a development image. - -Replace the contents of your Dockerfile with the following. - -```dockerfile {hl_lines="22-29"} -# syntax=docker/dockerfile:1 - -FROM eclipse-temurin:21-jdk-jammy as deps -WORKDIR /build -COPY --chmod=0755 mvnw mvnw -COPY .mvn/ .mvn/ -RUN --mount=type=bind,source=pom.xml,target=pom.xml \ - --mount=type=cache,target=/root/.m2 ./mvnw dependency:go-offline -DskipTests - -FROM deps as package -WORKDIR /build -COPY ./src src/ -RUN --mount=type=bind,source=pom.xml,target=pom.xml \ - --mount=type=cache,target=/root/.m2 \ - ./mvnw package -DskipTests && \ - mv target/$(./mvnw help:evaluate -Dexpression=project.artifactId -q -DforceStdout)-$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout).jar target/app.jar - -FROM package as extract -WORKDIR /build -RUN java -Djarmode=layertools -jar target/app.jar extract --destination target/extracted - -FROM extract as development -WORKDIR /build -RUN cp -r /build/target/extracted/dependencies/. ./ -RUN cp -r /build/target/extracted/spring-boot-loader/. ./ -RUN cp -r /build/target/extracted/snapshot-dependencies/. ./ -RUN cp -r /build/target/extracted/application/. ./ -ENV JAVA_TOOL_OPTIONS -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:8000 -CMD [ "java", "-Dspring.profiles.active=postgres", "org.springframework.boot.loader.launch.JarLauncher" ] - -FROM eclipse-temurin:21-jre-jammy AS final -ARG UID=10001 -RUN adduser \ - --disabled-password \ - --gecos "" \ - --home "/nonexistent" \ - --shell "/sbin/nologin" \ - --no-create-home \ - --uid "${UID}" \ - appuser -USER appuser -COPY --from=extract build/target/extracted/dependencies/ ./ -COPY --from=extract build/target/extracted/spring-boot-loader/ ./ -COPY --from=extract build/target/extracted/snapshot-dependencies/ ./ -COPY --from=extract build/target/extracted/application/ ./ -EXPOSE 8080 -ENTRYPOINT [ "java", "-Dspring.profiles.active=postgres", "org.springframework.boot.loader.launch.JarLauncher" ] -``` - -Save and close the `Dockerfile`. - -In the `Dockerfile` you added a new stage labeled `development` based on the `extract` stage. In this stage, you copy the extracted files to a common directory, then run a command to start the application. In the command, you expose port 8000 and declare the debug configuration for the JVM so that you can attach a debugger. - -## Use Compose to develop locally - -The current Compose file doesn't start your development container. To do that, you must update your Compose file to target the development stage. Also, update the port mapping of the server service to provide access for the debugger. - -Open the `docker-compose.yaml` and add the following instructions into the file. - -```yaml {hl_lines=["5","8"]} -services: - server: - build: - context: . - target: development - ports: - - 8080:8080 - - 8000:8000 - depends_on: - db: - condition: service_healthy - environment: - - POSTGRES_URL=jdbc:postgresql://db:5432/petclinic - db: - image: postgres:18 - restart: always - volumes: - - db-data:/var/lib/postgresql - environment: - - POSTGRES_DB=petclinic - - POSTGRES_USER=petclinic - - POSTGRES_PASSWORD=petclinic - ports: - - 5432:5432 - healthcheck: - test: ["CMD", "pg_isready", "-U", "petclinic"] - interval: 10s - timeout: 5s - retries: 5 -volumes: - db-data: -``` - -Now, start your application and to confirm that it's running. - -```console -$ docker compose up --build -``` - -Finally, test your API endpoint. Run the following curl command: - -```console -$ curl --request GET \ - --url http://localhost:8080/vets \ - --header 'content-type: application/json' -``` - -You should receive the following response: - -```json -{ - "vetList": [ - { - "id": 1, - "firstName": "James", - "lastName": "Carter", - "specialties": [], - "nrOfSpecialties": 0, - "new": false - }, - { - "id": 2, - "firstName": "Helen", - "lastName": "Leary", - "specialties": [{ "id": 1, "name": "radiology", "new": false }], - "nrOfSpecialties": 1, - "new": false - }, - { - "id": 3, - "firstName": "Linda", - "lastName": "Douglas", - "specialties": [ - { "id": 3, "name": "dentistry", "new": false }, - { "id": 2, "name": "surgery", "new": false } - ], - "nrOfSpecialties": 2, - "new": false - }, - { - "id": 4, - "firstName": "Rafael", - "lastName": "Ortega", - "specialties": [{ "id": 2, "name": "surgery", "new": false }], - "nrOfSpecialties": 1, - "new": false - }, - { - "id": 5, - "firstName": "Henry", - "lastName": "Stevens", - "specialties": [{ "id": 1, "name": "radiology", "new": false }], - "nrOfSpecialties": 1, - "new": false - }, - { - "id": 6, - "firstName": "Sharon", - "lastName": "Jenkins", - "specialties": [], - "nrOfSpecialties": 0, - "new": false - } - ] -} -``` - -## Connect a Debugger - -You’ll use the debugger that comes with the IntelliJ IDEA. You can use the community version of this IDE. Open your project in IntelliJ IDEA, go to the **Run** menu, and then **Edit Configuration**. Add a new Remote JVM Debug configuration similar to the following: - -![Java Connect a Debugger](images/connect-debugger.webp) - -Set a breakpoint. - -Open `src/main/java/org/springframework/samples/petclinic/vet/VetController.java` and add a breakpoint inside the `showResourcesVetList` function. - -To start your debug session, select the **Run** menu and then **Debug _NameOfYourConfiguration_**. - -![Debug menu](images/debug-menu.webp?w=300) - -You should now see the connection in the logs of your Compose application. - -![Compose log file ](images/compose-logs.webp) - -You can now call the server endpoint. - -```console -$ curl --request GET --url http://localhost:8080/vets -``` - -You should have seen the code break on the marked line and now you are able to use the debugger just like you would normally. You can also inspect and watch variables, set conditional breakpoints, view stack traces and a do bunch of other stuff. - -![Debugger code breakpoint](images/debugger-breakpoint.webp) - -Press `ctrl+c` in the terminal to stop your application. - -## Automatically update services - -Use Compose Watch to automatically update your running Compose services as you -edit and save your code. For more details about Compose Watch, see -[Use Compose Watch](/manuals/compose/how-tos/file-watch.md). - -Open your `docker-compose.yaml` file in an IDE or text editor and then add the -Compose Watch instructions. The following is the updated `docker-compose.yaml` -file. - -```yaml {hl_lines="14-17"} -services: - server: - build: - context: . - target: development - ports: - - 8080:8080 - - 8000:8000 - depends_on: - db: - condition: service_healthy - environment: - - POSTGRES_URL=jdbc:postgresql://db:5432/petclinic - develop: - watch: - - action: rebuild - path: . - db: - image: postgres:18 - restart: always - volumes: - - db-data:/var/lib/postgresql - environment: - - POSTGRES_DB=petclinic - - POSTGRES_USER=petclinic - - POSTGRES_PASSWORD=petclinic - ports: - - 5432:5432 - healthcheck: - test: ["CMD", "pg_isready", "-U", "petclinic"] - interval: 10s - timeout: 5s - retries: 5 -volumes: - db-data: -``` - -Run the following command to run your application with Compose Watch. - -```console -$ docker compose watch -``` - -Open a web browser and view the application at [http://localhost:8080](http://localhost:8080). You should see the Spring Pet Clinic home page. - -Any changes to the application's source files on your local machine will now be automatically reflected in the running container. - -Open `spring-petclinic/src/main/resources/templates/fragments/layout.html` in an IDE or text editor and update the `Home` navigation string by adding an exclamation mark. - -```diff --
  • -+
  • - -``` - -Save the changes to `layout.html` and then you can continue developing while the container automatically rebuilds. - -After the container is rebuilt and running, refresh [http://localhost:8080](http://localhost:8080) and then verify that **Home!** now appears in the menu. - -Press `ctrl+c` in the terminal to stop Compose Watch. - -## Summary - -In this section, you took a look at running a database locally and persisting the data. You also created a development image that contains the JDK and lets you attach a debugger. Finally, you set up your Compose file to expose the debugging port and configured Compose Watch to live reload your changes. - -Related information: - -- [Compose file reference](/reference/compose-file/) -- [Compose Watch](/manuals/compose/how-tos/file-watch.md) -- [Dockerfile reference](/reference/dockerfile/) - -## Next steps - -In the next section, you’ll take a look at how to run unit tests in Docker. diff --git a/content/guides/java/run-tests.md b/content/guides/java/run-tests.md deleted file mode 100644 index c88025ad82b6..000000000000 --- a/content/guides/java/run-tests.md +++ /dev/null @@ -1,125 +0,0 @@ ---- -title: Run your Java tests -linkTitle: Run your tests -weight: 30 -keywords: Java, build, test -description: How to build and run your Java tests -aliases: - - /language/java/run-tests/ - - /guides/language/java/run-tests/ ---- - -## Prerequisites - -Complete all the previous sections of this guide, starting with [Containerize a Java application](containerize.md). - -## Overview - -Testing is an essential part of modern software development. Testing can mean a lot of things to different development teams. There are unit tests, integration tests and end-to-end testing. In this guide you'll take a look at running your unit tests in Docker. - -### Multi-stage Dockerfile for testing - -In the following example, you'll pull the testing commands into your Dockerfile. -Replace the contents of your Dockerfile with the following. - -```dockerfile {hl_lines="3-19"} -# syntax=docker/dockerfile:1 - -FROM eclipse-temurin:21-jdk-jammy as base -WORKDIR /build -COPY --chmod=0755 mvnw mvnw -COPY .mvn/ .mvn/ - -FROM base as test -WORKDIR /build -COPY ./src src/ -RUN --mount=type=bind,source=pom.xml,target=pom.xml \ - --mount=type=cache,target=/root/.m2 \ - ./mvnw test - -FROM base as deps -WORKDIR /build -RUN --mount=type=bind,source=pom.xml,target=pom.xml \ - --mount=type=cache,target=/root/.m2 \ - ./mvnw dependency:go-offline -DskipTests - -FROM deps as package -WORKDIR /build -COPY ./src src/ -RUN --mount=type=bind,source=pom.xml,target=pom.xml \ - --mount=type=cache,target=/root/.m2 \ - ./mvnw package -DskipTests && \ - mv target/$(./mvnw help:evaluate -Dexpression=project.artifactId -q -DforceStdout)-$(./mvnw help:evaluate -Dexpression=project.version -q -DforceStdout).jar target/app.jar - -FROM package as extract -WORKDIR /build -RUN java -Djarmode=layertools -jar target/app.jar extract --destination target/extracted - -FROM extract as development -WORKDIR /build -RUN cp -r /build/target/extracted/dependencies/. ./ -RUN cp -r /build/target/extracted/spring-boot-loader/. ./ -RUN cp -r /build/target/extracted/snapshot-dependencies/. ./ -RUN cp -r /build/target/extracted/application/. ./ -ENV JAVA_TOOL_OPTIONS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:8000" -CMD [ "java", "-Dspring.profiles.active=postgres", "org.springframework.boot.loader.launch.JarLauncher" ] - -FROM eclipse-temurin:21-jre-jammy AS final -ARG UID=10001 -RUN adduser \ - --disabled-password \ - --gecos "" \ - --home "/nonexistent" \ - --shell "/sbin/nologin" \ - --no-create-home \ - --uid "${UID}" \ - appuser -USER appuser -COPY --from=extract build/target/extracted/dependencies/ ./ -COPY --from=extract build/target/extracted/spring-boot-loader/ ./ -COPY --from=extract build/target/extracted/snapshot-dependencies/ ./ -COPY --from=extract build/target/extracted/application/ ./ -EXPOSE 8080 -ENTRYPOINT [ "java", "-Dspring.profiles.active=postgres", "org.springframework.boot.loader.launch.JarLauncher" ] -``` - -First, you added a new base stage. In the base stage, you added common instructions that both the test and deps stage will need. - -Next, you added a new test stage labeled `test` based on the base stage. In this -stage you copied in the necessary source files and then specified `RUN` to run -`./mvnw test`. Instead of using `CMD`, you used `RUN` to run the tests. The -reason is that the `CMD` instruction runs when the container runs, and the `RUN` -instruction runs when the image is being built. When using `RUN`, the build will -fail if the tests fail. - -Finally, you updated the deps stage to be based on the base stage and removed -the instructions that are now in the base stage. - -Run the following command to build a new image using the test stage as the target and view the test results. Include `--progress=plain` to view the build output, `--no-cache` to ensure the tests always run, and `--target test` to target the test stage. - -Now, build your image and run your tests. You'll run the `docker build` command and add the `--target test` flag so that you specifically run the test build stage. - -```console -$ docker build -t java-docker-image-test --progress=plain --no-cache --target=test . -``` - -You should see output containing the following - -```console -... - -#15 101.3 [WARNING] Tests run: 45, Failures: 0, Errors: 0, Skipped: 2 -#15 101.3 [INFO] -#15 101.3 [INFO] ------------------------------------------------------------------------ -#15 101.3 [INFO] BUILD SUCCESS -#15 101.3 [INFO] ------------------------------------------------------------------------ -#15 101.3 [INFO] Total time: 01:39 min -#15 101.3 [INFO] Finished at: 2024-02-01T23:24:48Z -#15 101.3 [INFO] ------------------------------------------------------------------------ -#15 DONE 101.4s -``` - -## Next steps - -In the next section, you’ll take a look at how to set up a CI/CD pipeline using -GitHub Actions. diff --git a/content/guides/jupyter.md b/content/guides/jupyter.md index b5620cba0aa3..699f2186a6d2 100644 --- a/content/guides/jupyter.md +++ b/content/guides/jupyter.md @@ -5,11 +5,10 @@ title: Data science with JupyterLab toc_max: 2 summary: | Use Docker to run Jupyter notebooks. -tags: [data-science] -languages: [python] aliases: - /guides/use-case/jupyter/ params: + tags: [ai] time: 20 minutes --- diff --git a/content/guides/kafka.md b/content/guides/kafka.md index 78153a04f7ed..17b3dee95e2e 100644 --- a/content/guides/kafka.md +++ b/content/guides/kafka.md @@ -5,11 +5,10 @@ title: Developing event-driven applications with Kafka and Docker linktitle: Event-driven apps with Kafka summary: | This guide explains how to run Apache Kafka in Docker containers. -tags: [distributed-systems] -languages: [js] aliases: - /guides/use-case/kafka/ params: + tags: [deployment] time: 20 minutes --- diff --git a/content/guides/kube-deploy.md b/content/guides/kube-deploy.md index c4b535385365..eeef1ba9b9ef 100644 --- a/content/guides/kube-deploy.md +++ b/content/guides/kube-deploy.md @@ -7,8 +7,8 @@ aliases: - /guides/deployment-orchestration/kube-deploy/ summary: | Learn how to deploy and orchestrate Docker containers using Kubernetes. -tags: [deploy] params: + tags: [deployment] time: 10 minutes --- diff --git a/content/guides/lab-agentic-apps.md b/content/guides/lab-agentic-apps.md index 2e8d5e5a246f..5839ecd44bcc 100644 --- a/content/guides/lab-agentic-apps.md +++ b/content/guides/lab-agentic-apps.md @@ -11,14 +11,8 @@ keywords: AI, Docker, Model Runner, MCP Gateway, agentic apps, lab, labspace aliases: - /labs/docker-for-ai/agentic-apps/ params: - tags: [ai, labs] + tags: [ai] time: 20 minutes - resource_links: - - title: Docker Model Runner docs - url: /ai/model-runner/ - - title: Docker MCP Gateway docs - url: /ai/mcp-gateway/ - - title: Labspace repository url: https://github.com/dockersamples/labspace-agentic-apps-with-docker --- diff --git a/content/guides/lab-ai-fundamentals.md b/content/guides/lab-ai-fundamentals.md index 02837f1774a3..01163b62c1e6 100644 --- a/content/guides/lab-ai-fundamentals.md +++ b/content/guides/lab-ai-fundamentals.md @@ -12,12 +12,8 @@ keywords: AI, Docker, Model Runner, prompt engineering, RAG, tool calling, lab, aliases: - /labs/docker-for-ai/ai-fundamentals/ params: - tags: [ai, labs] + tags: [ai] time: 45 minutes - resource_links: - - title: Docker Model Runner docs - url: /ai/model-runner/ - - title: Labspace repository url: https://github.com/dockersamples/labspace-ai-fundamentals --- diff --git a/content/guides/lab-attestation-basics.md b/content/guides/lab-attestation-basics.md index 7c9aa42aa1bf..3560d7a53fe4 100644 --- a/content/guides/lab-attestation-basics.md +++ b/content/guides/lab-attestation-basics.md @@ -10,16 +10,8 @@ summary: | attach OpenVEX statements to declare vulnerability exploitability status. keywords: Docker, supply chain, SBOM, provenance, SLSA, Cosign, VEX, attestations, security, lab, labspace params: - tags: [labs] + tags: [security] time: 45 minutes - resource_links: - - title: Build attestations - url: /build/metadata/attestations/ - - title: SBOM attestations - url: /build/metadata/attestations/sbom/ - - title: Provenance attestations - url: /build/metadata/attestations/slsa-provenance/ - - title: Labspace repository url: https://github.com/dockersamples/labspace-attestation-basics --- diff --git a/content/guides/lab-building-images.md b/content/guides/lab-building-images.md index 550e4bb90fbd..6eeef74b76c7 100644 --- a/content/guides/lab-building-images.md +++ b/content/guides/lab-building-images.md @@ -11,16 +11,8 @@ summary: | base image selection, and build secrets. keywords: Docker, Dockerfile, images, multi-stage builds, layer caching, build secrets, lab, labspace params: - tags: [labs] + tags: [cicd] time: 45 minutes - resource_links: - - title: Dockerfile reference - url: /reference/dockerfile/ - - title: Multi-stage builds - url: /build/building/multi-stage/ - - title: Build secrets - url: /build/building/secrets/ - - title: Labspace repository url: https://github.com/dockersamples/labspace-building-images --- diff --git a/content/guides/lab-compose-quickstart.md b/content/guides/lab-compose-quickstart.md index 17605a66aef0..67f4a43b582e 100644 --- a/content/guides/lab-compose-quickstart.md +++ b/content/guides/lab-compose-quickstart.md @@ -11,14 +11,8 @@ summary: | with watch mode, data persistence, and modular Compose file composition. keywords: Docker, Compose, multi-container, Flask, Redis, watch mode, volumes, lab, labspace params: - tags: [labs] + tags: [cicd] time: 45 minutes - resource_links: - - title: Docker Compose docs - url: /compose/ - - title: Compose watch mode - url: /compose/how-tos/file-watch/ - - title: Labspace repository url: https://github.com/dockersamples/labspace-compose-quickstart --- diff --git a/content/guides/lab-container-getting-started.md b/content/guides/lab-container-getting-started.md index e3499688d3e5..fb40a4884a03 100644 --- a/content/guides/lab-container-getting-started.md +++ b/content/guides/lab-container-getting-started.md @@ -9,14 +9,8 @@ summary: | image from a Node.js app, and optionally push it to Docker Hub. keywords: Docker, containers, Dockerfile, images, getting started, lab, labspace params: - tags: [labs] + tags: [cicd] time: 30 minutes - resource_links: - - title: Docker overview - url: /get-started/docker-overview/ - - title: Dockerfile reference - url: /reference/dockerfile/ - - title: Labspace repository url: https://github.com/dockersamples/labspace-container-getting-started --- diff --git a/content/guides/lab-container-supported-development.md b/content/guides/lab-container-supported-development.md index 9a8b526023a9..1ee08231940b 100644 --- a/content/guides/lab-container-supported-development.md +++ b/content/guides/lab-container-supported-development.md @@ -11,14 +11,8 @@ summary: | visualizer — all without installing anything on the host. keywords: Docker, Compose, local development, PostgreSQL, pgAdmin, containers, lab, labspace params: - tags: [labs] + tags: [cicd] time: 30 minutes - resource_links: - - title: Docker Compose docs - url: /compose/ - - title: Bind mounts - url: /engine/storage/bind-mounts/ - - title: Labspace repository url: https://github.com/dockersamples/labspace-container-supported-development --- diff --git a/content/guides/lab-containerized-sdlc.md b/content/guides/lab-containerized-sdlc.md index 7e26ea96c2d0..b4f68a1ecc4e 100644 --- a/content/guides/lab-containerized-sdlc.md +++ b/content/guides/lab-containerized-sdlc.md @@ -11,14 +11,8 @@ summary: | with containers at every stage of the SDLC. keywords: Docker, Compose, Testcontainers, Kubernetes, CI/CD, SDLC, lab, labspace params: - tags: [labs] + tags: [cicd] time: 60 minutes - resource_links: - - title: Docker Compose docs - url: /compose/ - - title: Testcontainers docs - url: https://testcontainers.com/ - - title: Labspace repository url: https://github.com/dockersamples/labspace-containerized-sdlc --- diff --git a/content/guides/lab-creating-ai-product-reviewer.md b/content/guides/lab-creating-ai-product-reviewer.md index af02981645b1..c1476d1f1e15 100644 --- a/content/guides/lab-creating-ai-product-reviewer.md +++ b/content/guides/lab-creating-ai-product-reviewer.md @@ -13,12 +13,8 @@ keywords: AI, Docker, Model Runner, sentiment analysis, embeddings, RAG, lab, la aliases: - /labs/docker-for-ai/creating-ai-product-reviewer/ params: - tags: [ai, labs] + tags: [ai] time: 60 minutes - resource_links: - - title: Docker Model Runner docs - url: /ai/model-runner/ - - title: Labspace repository url: https://github.com/dockersamples/labspace-creating-ai-product-reviewer --- diff --git a/content/guides/lab-dhi-node.md b/content/guides/lab-dhi-node.md index 3e775b7e3a9d..6906195c00ef 100644 --- a/content/guides/lab-dhi-node.md +++ b/content/guides/lab-dhi-node.md @@ -11,16 +11,8 @@ summary: | builds with DHI, and explore SBOMs, VEX, and compliance attestations. keywords: Docker, Hardened Images, DHI, Node.js, Docker Scout, CVE, security, SBOM, lab, labspace params: - tags: [labs, dhi] + tags: [security] time: 30 minutes - resource_links: - - title: Docker Hardened Images - url: /dhi/ - - title: Docker Scout docs - url: /scout/ - - title: Build attestations - url: /build/metadata/attestations/ - - title: Labspace repository url: https://github.com/dockersamples/labspace-dhi-node --- diff --git a/content/guides/lab-docker-agent.md b/content/guides/lab-docker-agent.md index bb2cfcadbb70..4fce0f713515 100644 --- a/content/guides/lab-docker-agent.md +++ b/content/guides/lab-docker-agent.md @@ -12,14 +12,8 @@ aliases: - /labs/docker-for-ai/cagent/ - /guides/lab-cagent/ params: - tags: [ai, labs] + tags: [ai] time: 20 minutes - resource_links: - - title: Docker Agent documentation - url: https://github.com/docker/docker-agent - - title: Docker MCP Toolkit - url: https://docs.docker.com/ai/mcp-catalog-and-toolkit/toolkit/ - - title: Labspace repository url: https://github.com/ajeetraina/labspace-cagent --- diff --git a/content/guides/lab-docker-for-ai-redirect.md b/content/guides/lab-docker-for-ai-redirect.md deleted file mode 100644 index 0851380aef86..000000000000 --- a/content/guides/lab-docker-for-ai-redirect.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -title: Docker for AI Labs -keywords: docker, ai, labs, machine learning, hands-on -type: redirect -target: /guides/?tags=labs -aliases: - - /labs/ - - /labs/docker-for-ai/ ---- diff --git a/content/guides/lab-mcp-gateway.md b/content/guides/lab-mcp-gateway.md index 4d53c8f37a4b..28b6cc81ef5a 100644 --- a/content/guides/lab-mcp-gateway.md +++ b/content/guides/lab-mcp-gateway.md @@ -11,14 +11,8 @@ keywords: AI, Docker, MCP, MCP Gateway, MCP servers, lab, labspace aliases: - /labs/docker-for-ai/mcp-gateway/ params: - tags: [ai, labs] + tags: [ai] time: 20 minutes - resource_links: - - title: Docker MCP Gateway docs - url: /ai/mcp-gateway/ - - title: MCP Gateway GitHub - url: https://github.com/docker/mcp-gateway - - title: Labspace repository url: https://github.com/dockersamples/labspace-mcp-gateway --- diff --git a/content/guides/language-translation.md b/content/guides/language-translation.md index 751230ce4ade..5e569c2f8fa7 100644 --- a/content/guides/language-translation.md +++ b/content/guides/language-translation.md @@ -6,11 +6,10 @@ description: Learn how to build and run a language translation application using summary: | This guide demonstrates how to use Docker to deploy language translation models for NLP tasks. -tags: [ai] -languages: [python] aliases: - /guides/use-case/nlp/language-translation/ params: + tags: [ai] time: 20 minutes --- diff --git a/content/guides/localstack.md b/content/guides/localstack.md index 5bc2bd87034f..4f7a0995947e 100644 --- a/content/guides/localstack.md +++ b/content/guides/localstack.md @@ -6,9 +6,8 @@ linktitle: AWS development with LocalStack summary: | This guide explains how to use Docker to run LocalStack, a local AWS cloud stack emulator. -tags: [cloud-services] -languages: [js] params: + tags: [databases] time: 20 minutes --- diff --git a/content/guides/named-entity-recognition.md b/content/guides/named-entity-recognition.md index 82b4149a9e88..b2cc582597c6 100644 --- a/content/guides/named-entity-recognition.md +++ b/content/guides/named-entity-recognition.md @@ -6,11 +6,10 @@ description: Learn how to build and run a named entity recognition application u summary: | This guide explains how to containerize named entity recognition (NER) models using Docker. -tags: [ai] -languages: [python] aliases: - /guides/use-case/nlp/named-entity-recognition/ params: + tags: [ai] time: 20 minutes --- diff --git a/content/guides/nextjs/_index.md b/content/guides/nextjs/_index.md index 995744c5c099..736dbc397230 100644 --- a/content/guides/nextjs/_index.md +++ b/content/guides/nextjs/_index.md @@ -7,14 +7,18 @@ summary: | This guide explains how to containerize Next.js applications, set up development and testing in containers, automate builds with GitHub Actions, and deploy to Kubernetes. -toc_min: 1 -toc_max: 2 -languages: [js] -tags: [frameworks] +aliases: + - /guides/nextjs/configure-github-actions/ + - /guides/nextjs/containerize/ + - /guides/nextjs/deploy/ + - /guides/nextjs/develop/ + - /guides/nextjs/run-tests/ params: + tags: [cicd] time: 20 minutes --- + This guide shows you how to containerize a Next.js application using Docker, following best practices for creating efficient, production-ready containers. [Next.js](https://nextjs.org/) is a React framework that enables server-side @@ -52,3 +56,1941 @@ Before you begin, make sure you're familiar with the following: - Understanding of Docker concepts such as images, containers, and Dockerfiles. If you're new to Docker, start with the [Docker basics](/get-started/docker-concepts/the-basics/what-is-a-container.md) guide. Once you've completed the Next.js getting started modules, you'll be ready to containerize your own Next.js application using the examples and instructions provided in this guide. + +## Containerize a Next.js Application + +### Prerequisites + +Before you begin, make sure the following tools are installed and available on your system: + +- You have installed the latest version of [Docker Desktop](/get-started/get-docker.md). +- You have a [git client](https://git-scm.com/downloads). The examples in this section use a command-line based git client, but you can use any client. + +> [!NOTE] +> New to Docker? Start with the [Docker basics](/get-started/docker-concepts/the-basics/what-is-a-container.md) guide to get familiar with key concepts like images, containers, and Dockerfiles. + +--- + +### Overview + +This guide walks you through containerizing a Next.js application with Docker. +You'll learn how to create a production-ready Docker image using best +practices that improve performance, security, scalability, and deployment +efficiency. + +By the end of this guide, you will: + +- Containerize a Next.js application using Docker. +- Create and optimize a Dockerfile for production builds. +- Use multi-stage builds to minimize image size. +- Leverage Next.js standalone or export output for efficient containerization. +- Follow best practices for building secure and maintainable Docker images. + +--- + +### Get the sample application + +Clone the sample application to use with this guide. Open a terminal, change +directory to a directory that you want to work in, and run the following command +to clone the git repository: + +```console +$ git clone https://github.com/kristiyan-velkov/docker-nextjs-sample +``` +--- + +### Build the Docker image + +Next.js has specific requirements for production deployments. This guide shows two approaches: `standalone` output (Node.js server) and `export` output (static files with Nginx). + +> [!TIP] +> +> [Gordon](/ai/gordon/), Docker's AI assistant, can generate Docker assets for your project. Ask Gordon to create a Dockerfile, Compose file, and `.dockerignore` tailored to your application. + +#### Step 1: Configure Next.js and create the Dockerfile + +Before creating a Dockerfile, choose a base image: the [Node.js Official Image](https://hub.docker.com/_/node) or a [Docker Hardened Image (DHI)](https://hub.docker.com/hardened-images/catalog) from the Hardened Image catalog. Choosing DHI gives you a production-ready, lightweight, and secure image. For more information, see [Docker Hardened Images](https://docs.docker.com/dhi/). + +> [!IMPORTANT] +> This guide uses stable Node.js LTS image tags that are considered secure when the guide is written. Because new releases and security patches are published regularly, always review the [official Node.js Docker images](https://hub.docker.com/_/node) and select a secure, up-to-date version before building or deploying. + +--- + +##### 1.1 Next.js with standalone output + +Standalone output (`output: "standalone"`) makes Next.js build a self-contained output that includes only the files and dependencies needed to run the application. A single `node server.js` can serve the app, which is ideal for Docker and supports server-side rendering, API routes, and incremental static regeneration. For details, see the [Next.js output configuration documentation](https://nextjs.org/docs/app/api-reference/config/next-config-js/output) (including the "standalone" option). + +The container runs the Next.js server with Node.js on port 3000. + +Configure Next.js — Open or create `next.config.ts` in your project root: + +```ts +import type { NextConfig } from "next"; + +const nextConfig: NextConfig = { + output: "standalone", +}; + +export default nextConfig; +``` + +Choose either a Docker Hardened Image or the Docker Official Image, then create a `Dockerfile` using the content from the selected tab below. + +{{< tabs >}} +{{< tab name="Using Docker Hardened Images" >}} + +Docker Hardened Images (DHIs) are available for Node.js in the [Docker Hardened Images catalog](https://hub.docker.com/hardened-images/catalog/dhi/node). For more information, see the [DHI quickstart](/dhi/get-started/) guide. + +1. Sign in to the DHI registry: + ```console + $ docker login dhi.io + ``` + +2. Pull the Node.js DHI (check the catalog for available versions): + ```console + $ docker pull dhi.io/node:24-alpine3.22-dev + ``` + +3. Create a file named `Dockerfile` with the following contents. The `FROM` instructions use `dhi.io/node:24-alpine3.22-dev`. Check the [Docker Hardened Images catalog](https://hub.docker.com/hardened-images/catalog) for the latest versions and update the image tags as needed for security and compatibility. + + ```dockerfile + # ============================================ + # Stage 1: Dependencies Installation Stage + # ============================================ + + # IMPORTANT: Docker Hardened Image (DHI) Version Maintenance + # This Dockerfile uses dhi.io/node. Regularly validate and update to the latest DHI versions in the catalog for security and compatibility. + + FROM dhi.io/node:24-alpine3.22-dev AS dependencies + + # Set working directory + WORKDIR /app + + # Copy package-related files first to leverage Docker's caching mechanism + COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml* .npmrc* ./ + + # Install project dependencies with frozen lockfile for reproducible builds + RUN --mount=type=cache,target=/root/.npm \ + --mount=type=cache,target=/usr/local/share/.cache/yarn \ + --mount=type=cache,target=/root/.local/share/pnpm/store \ + if [ -f package-lock.json ]; then \ + npm ci --no-audit --no-fund; \ + elif [ -f yarn.lock ]; then \ + corepack enable yarn && yarn install --frozen-lockfile --production=false; \ + elif [ -f pnpm-lock.yaml ]; then \ + corepack enable pnpm && pnpm install --frozen-lockfile; \ + else \ + echo "No lockfile found." && exit 1; \ + fi + + # ============================================ + # Stage 2: Build Next.js application in standalone mode + # ============================================ + + FROM dhi.io/node:24-alpine3.22-dev AS builder + + # Set working directory + WORKDIR /app + + # Copy project dependencies from dependencies stage + COPY --from=dependencies /app/node_modules ./node_modules + + # Copy application source code + COPY . . + + ENV NODE_ENV=production + + # Next.js collects completely anonymous telemetry data about general usage. + # Learn more here: https://nextjs.org/telemetry + # Uncomment the following line in case you want to disable telemetry during the build. + # ENV NEXT_TELEMETRY_DISABLED=1 + + # Build Next.js application + # If you want to speed up Docker rebuilds, you can cache the build artifacts + # by adding: --mount=type=cache,target=/app/.next/cache + # This caches the .next/cache directory across builds, but it also prevents + # .next/cache/fetch-cache from being included in the final image, meaning + # cached fetch responses from the build won't be available at runtime. + RUN if [ -f package-lock.json ]; then \ + npm run build; \ + elif [ -f yarn.lock ]; then \ + corepack enable yarn && yarn build; \ + elif [ -f pnpm-lock.yaml ]; then \ + corepack enable pnpm && pnpm build; \ + else \ + echo "No lockfile found." && exit 1; \ + fi + + # ============================================ + # Stage 3: Run Next.js application + # ============================================ + + FROM dhi.io/node:24-alpine3.22-dev AS runner + + # Set working directory + WORKDIR /app + + # Set production environment variables + ENV NODE_ENV=production + ENV PORT=3000 + ENV HOSTNAME="0.0.0.0" + + # Next.js collects completely anonymous telemetry data about general usage. + # Learn more here: https://nextjs.org/telemetry + # Uncomment the following line in case you want to disable telemetry during the run time. + # ENV NEXT_TELEMETRY_DISABLED=1 + + # Copy production assets + COPY --from=builder --chown=node:node /app/public ./public + + # Set the correct permission for prerender cache + RUN mkdir .next + RUN chown node:node .next + + # Automatically leverage output traces to reduce image size + # https://nextjs.org/docs/advanced-features/output-file-tracing + COPY --from=builder --chown=node:node /app/.next/standalone ./ + COPY --from=builder --chown=node:node /app/.next/static ./.next/static + + # If you want to persist the fetch cache generated during the build so that + # cached responses are available immediately on startup, uncomment this line: + # COPY --from=builder --chown=node:node /app/.next/cache ./.next/cache + + # Switch to non-root user for security best practices + USER node + + # Expose port 3000 to allow HTTP traffic + EXPOSE 3000 + + # Start Next.js standalone server + CMD ["node", "server.js"] + ``` + +{{< /tab >}} +{{< tab name="Using the Docker Official Image" >}} + +Create a file named `Dockerfile` with the following contents (uses `node`): + +```dockerfile + # ============================================ + # Stage 1: Dependencies Installation Stage + # ============================================ + + ARG NODE_VERSION=24.14.0-slim + + FROM node:${NODE_VERSION} AS dependencies + + # Set working directory + WORKDIR /app + + # Copy package-related files first to leverage Docker's caching mechanism + COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml* .npmrc* ./ + + # Install project dependencies with frozen lockfile for reproducible builds + RUN --mount=type=cache,target=/root/.npm \ + --mount=type=cache,target=/usr/local/share/.cache/yarn \ + --mount=type=cache,target=/root/.local/share/pnpm/store \ + if [ -f package-lock.json ]; then \ + npm ci --no-audit --no-fund; \ + elif [ -f yarn.lock ]; then \ + corepack enable yarn && yarn install --frozen-lockfile --production=false; \ + elif [ -f pnpm-lock.yaml ]; then \ + corepack enable pnpm && pnpm install --frozen-lockfile; \ + else \ + echo "No lockfile found." && exit 1; \ + fi + + # ============================================ + # Stage 2: Build Next.js application in standalone mode + # ============================================ + + FROM node:${NODE_VERSION} AS builder + + # Set working directory + WORKDIR /app + + # Copy project dependencies from dependencies stage + COPY --from=dependencies /app/node_modules ./node_modules + + # Copy application source code + COPY . . + + ENV NODE_ENV=production + + # Next.js collects completely anonymous telemetry data about general usage. + # Learn more here: https://nextjs.org/telemetry + # Uncomment the following line in case you want to disable telemetry during the build. + # ENV NEXT_TELEMETRY_DISABLED=1 + + # Build Next.js application + # If you want to speed up Docker rebuilds, you can cache the build artifacts + # by adding: --mount=type=cache,target=/app/.next/cache + # This caches the .next/cache directory across builds, but it also prevents + # .next/cache/fetch-cache from being included in the final image, meaning + # cached fetch responses from the build won't be available at runtime. + RUN if [ -f package-lock.json ]; then \ + npm run build; \ + elif [ -f yarn.lock ]; then \ + corepack enable yarn && yarn build; \ + elif [ -f pnpm-lock.yaml ]; then \ + corepack enable pnpm && pnpm build; \ + else \ + echo "No lockfile found." && exit 1; \ + fi + + # ============================================ + # Stage 3: Run Next.js application + # ============================================ + + FROM node:${NODE_VERSION} AS runner + + # Set working directory + WORKDIR /app + + # Set production environment variables + ENV NODE_ENV=production + ENV PORT=3000 + ENV HOSTNAME="0.0.0.0" + + # Next.js collects completely anonymous telemetry data about general usage. + # Learn more here: https://nextjs.org/telemetry + # Uncomment the following line in case you want to disable telemetry during the run time. + # ENV NEXT_TELEMETRY_DISABLED=1 + + # Copy production assets + COPY --from=builder --chown=node:node /app/public ./public + + # Set the correct permission for prerender cache + RUN mkdir .next + RUN chown node:node .next + + # Automatically leverage output traces to reduce image size + # https://nextjs.org/docs/advanced-features/output-file-tracing + COPY --from=builder --chown=node:node /app/.next/standalone ./ + COPY --from=builder --chown=node:node /app/.next/static ./.next/static + + # If you want to persist the fetch cache generated during the build so that + # cached responses are available immediately on startup, uncomment this line: + # COPY --from=builder --chown=node:node /app/.next/cache ./.next/cache + + # Switch to non-root user for security best practices + USER node + + # Expose port 3000 to allow HTTP traffic + EXPOSE 3000 + + # Start Next.js standalone server + CMD ["node", "server.js"] +``` + +> [!NOTE] +> This Dockerfile uses three stages: `dependencies`, `builder`, and `runner`. The final image runs `node server.js` and listens on port 3000. + +{{< /tab >}} +{{< /tabs >}} + +--- + +##### 1.2 Next.js with export output + +Output export (`output: "export"`) makes Next.js build a fully static site at build time. It generates HTML, CSS, and JavaScript into an `out` directory that can be served by any static host or CDN—no Node.js server at runtime. Use this when you don't need server-side rendering or API routes. For details, see the [Next.js output configuration documentation](https://nextjs.org/docs/app/api-reference/config/next-config-js/output). + +Configure Next.js — Open `next.config.ts` in your project root and add the following code: + +```ts +import type { NextConfig } from "next"; + +const nextConfig: NextConfig = { + output: "export", + trailingSlash: true, + images: { + unoptimized: true, + }, +}; + +export default nextConfig; +``` + +Choose either a Docker Hardened Image or the Docker Official Image, then create a `Dockerfile` using the content from the selected tab below. + +{{< tabs >}} +{{< tab name="Using Docker Hardened Images" >}} + +Docker Hardened Images (DHIs) are available for Node.js and Nginx in the [Docker Hardened Images catalog](https://hub.docker.com/hardened-images/catalog). For more information, see the [DHI quickstart](/dhi/get-started/) guide. + +1. Sign in to the DHI registry: + ```console + $ docker login dhi.io + ``` + +2. Pull the Node.js DHI (check the catalog for available versions): + ```console + $ docker pull dhi.io/node:24-alpine3.22-dev + ``` + +3. Pull the Nginx DHI (check the catalog for available versions): + ```console + $ docker pull dhi.io/nginx:1.28.0-alpine3.21-dev + ``` + +4. Create a file named `Dockerfile` with the following contents. The `FROM` instructions use Docker Hardened Images: `dhi.io/node:24-alpine3.22-dev` and `dhi.io/nginx:1.28.0-alpine3.21-dev`. Check the [Docker Hardened Images catalog](https://hub.docker.com/hardened-images/catalog) for the latest versions and update the image tags as needed for security and compatibility. + + ```dockerfile + # ============================================ + # Stage 1: Dependencies Installation Stage + # ============================================ + + # IMPORTANT: Docker Hardened Image (DHI) Version Maintenance + # This Dockerfile uses dhi.io/node and dhi.io/nginx. Regularly validate and update to the latest DHI versions in the catalog for security and compatibility. + + FROM dhi.io/node:24-alpine3.22-dev AS dependencies + + # Set the working directory + WORKDIR /app + + # Copy package-related files first to leverage Docker's caching mechanism + COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml* .npmrc* ./ + + # Install project dependencies with frozen lockfile for reproducible builds + RUN --mount=type=cache,target=/root/.npm \ + --mount=type=cache,target=/usr/local/share/.cache/yarn \ + --mount=type=cache,target=/root/.local/share/pnpm/store \ + if [ -f package-lock.json ]; then \ + npm ci --no-audit --no-fund; \ + elif [ -f yarn.lock ]; then \ + corepack enable yarn && yarn install --frozen-lockfile --production=false; \ + elif [ -f pnpm-lock.yaml ]; then \ + corepack enable pnpm && pnpm install --frozen-lockfile; \ + else \ + echo "No lockfile found." && exit 1; \ + fi + + # ============================================ + # Stage 2: Build Next.js Application + # ============================================ + + FROM dhi.io/node:24-alpine3.22-dev AS builder + + # Set the working directory + WORKDIR /app + + # Copy project dependencies from dependencies stage + COPY --from=dependencies /app/node_modules ./node_modules + + # Copy application source code + COPY . . + + ENV NODE_ENV=production + + # Next.js collects completely anonymous telemetry data about general usage. + # Learn more here: https://nextjs.org/telemetry + # Uncomment the following line in case you want to disable telemetry during the build. + # ENV NEXT_TELEMETRY_DISABLED=1 + + # Build Next.js application + RUN --mount=type=cache,target=/app/.next/cache \ + if [ -f package-lock.json ]; then \ + npm run build; \ + elif [ -f yarn.lock ]; then \ + corepack enable yarn && yarn build; \ + elif [ -f pnpm-lock.yaml ]; then \ + corepack enable pnpm && pnpm build; \ + else \ + echo "No lockfile found." && exit 1; \ + fi + + # ========================================= + # Stage 3: Serve Static Files with Nginx + # ========================================= + + FROM dhi.io/nginx:1.28.0-alpine3.21-dev AS runner + + # Set the working directory + WORKDIR /app + + # Next.js collects completely anonymous telemetry data about general usage. + # Learn more here: https://nextjs.org/telemetry + # Uncomment the following line in case you want to disable telemetry during the run time. + # ENV NEXT_TELEMETRY_DISABLED=1 + + # Copy custom Nginx config + COPY nginx.conf /etc/nginx/nginx.conf + + # Copy the static build output from the build stage to Nginx's default HTML serving directory + COPY --chown=nginx:nginx --from=builder /app/out /usr/share/nginx/html + + # Non-root user for security best practices + USER nginx + + # Expose port 8080 to allow HTTP traffic + EXPOSE 8080 + + # Start Nginx directly with custom config + ENTRYPOINT ["nginx", "-c", "/etc/nginx/nginx.conf"] + CMD ["-g", "daemon off;"] + ``` + +{{< /tab >}} +{{< tab name="Using the Docker Official Image" >}} + +Create a file named `Dockerfile` with the following contents (uses `node` and `nginxinc/nginx-unprivileged`): + +```dockerfile +# ============================================ +# Stage 1: Dependencies Installation Stage +# ============================================ + +ARG NODE_VERSION=24.14.0-slim +ARG NGINXINC_IMAGE_TAG=alpine3.22 + +FROM node:${NODE_VERSION} AS dependencies + +# Set the working directory +WORKDIR /app + +# Copy package-related files first to leverage Docker's caching mechanism +COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml* .npmrc* ./ + +# Install project dependencies with frozen lockfile for reproducible builds +RUN --mount=type=cache,target=/root/.npm \ + --mount=type=cache,target=/usr/local/share/.cache/yarn \ + --mount=type=cache,target=/root/.local/share/pnpm/store \ + if [ -f package-lock.json ]; then \ + npm ci --no-audit --no-fund; \ + elif [ -f yarn.lock ]; then \ + corepack enable yarn && yarn install --frozen-lockfile --production=false; \ + elif [ -f pnpm-lock.yaml ]; then \ + corepack enable pnpm && pnpm install --frozen-lockfile; \ + else \ + echo "No lockfile found." && exit 1; \ + fi + +# ============================================ +# Stage 2: Build Next.js Application +# ============================================ + +FROM node:${NODE_VERSION} AS builder + +# Set the working directory +WORKDIR /app + +# Copy project dependencies from dependencies stage +COPY --from=dependencies /app/node_modules ./node_modules + +# Copy application source code +COPY . . + +ENV NODE_ENV=production + +# Next.js collects completely anonymous telemetry data about general usage. +# Learn more here: https://nextjs.org/telemetry +# Uncomment the following line in case you want to disable telemetry during the build. +# ENV NEXT_TELEMETRY_DISABLED=1 + +# Build Next.js application +RUN --mount=type=cache,target=/app/.next/cache \ + if [ -f package-lock.json ]; then \ + npm run build; \ + elif [ -f yarn.lock ]; then \ + corepack enable yarn && yarn build; \ + elif [ -f pnpm-lock.yaml ]; then \ + corepack enable pnpm && pnpm build; \ + else \ + echo "No lockfile found." && exit 1; \ + fi + +# ========================================= +# Stage 3: Serve Static Files with Nginx +# ========================================= + +FROM nginxinc/nginx-unprivileged:${NGINXINC_IMAGE_TAG} AS runner + +# Set the working directory +WORKDIR /app + +# Next.js collects completely anonymous telemetry data about general usage. +# Learn more here: https://nextjs.org/telemetry +# Uncomment the following line in case you want to disable telemetry during the run time. +# ENV NEXT_TELEMETRY_DISABLED=1 + +# Copy custom Nginx config +COPY nginx.conf /etc/nginx/nginx.conf + +# Copy the static build output from the build stage to Nginx's default HTML serving directory +COPY --from=builder /app/out /usr/share/nginx/html + +# Non-root user for security best practices +USER nginx + +# Expose port 8080 to allow HTTP traffic +EXPOSE 8080 + +# Start Nginx directly with custom config +ENTRYPOINT ["nginx", "-c", "/etc/nginx/nginx.conf"] +CMD ["-g", "daemon off;"] +``` + +> [!NOTE] +> This guide uses [nginx-unprivileged](https://hub.docker.com/r/nginxinc/nginx-unprivileged) instead of the standard Nginx image to run as a non-root user, following security best practices. + +{{< /tab >}} +{{< /tabs >}} + +1. Create `nginx.conf` (required for export output only) — Create a file named `nginx.conf` in the root of your project: + + ```nginx + # Minimal Nginx config for static Next.js app + worker_processes 1; + + # Store PID in /tmp (always writable) + pid /tmp/nginx.pid; + + events { + worker_connections 1024; + } + + http { + include /etc/nginx/mime.types; + default_type application/octet-stream; + + # Disable logging to avoid permission issues + access_log off; + error_log /dev/stderr; + + # Optimize static file serving + sendfile on; + tcp_nopush on; + tcp_nodelay on; + keepalive_timeout 65; + + # Gzip compression + gzip on; + gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript; + gzip_min_length 256; + + server { + listen 8080; + server_name localhost; + + # Serve static files + root /usr/share/nginx/html; + index index.html; + + # Handle Next.js static export routing + # See: https://nextjs.org/docs/app/guides/static-exports#deploying + location / { + try_files $uri $uri.html $uri/ =404; + } + + # This is necessary when `trailingSlash: false` (default). + # You can omit this when `trailingSlash: true` in next.config. + # Handles nested routes like /blog/post -> /blog/post.html + location ~ ^/(.+)/$ { + rewrite ^/(.+)/$ /$1.html break; + } + + # Serve Next.js static assets + location ~ ^/_next/ { + try_files $uri =404; + expires 1y; + add_header Cache-Control "public, immutable"; + } + + # Optional 404 handling + error_page 404 /404.html; + location = /404.html { + internal; + } + } + } + ``` + + > [!NOTE] + > Export uses port 8080. For more details, see the [Next.js output configuration](https://nextjs.org/docs/app/api-reference/config/next-config-js/output) and [Nginx documentation](https://nginx.org/en/docs/). + +#### Step 2: Create the compose.yaml file + +Create a file named `compose.yaml` with the following contents: + +```yaml {collapse=true,title=compose.yaml} +services: + server: + build: + context: . + ports: + - 3000:3000 +``` + +> [!NOTE] +> If using export output (Nginx), change the port mapping to `8080:8080`. + +#### Step 3: Create the .dockerignore file + +The `.dockerignore` file tells Docker which files and folders to exclude when building the image. + + +> [!NOTE] +>This helps: +>- Reduce image size +>- Speed up the build process +>- Prevent sensitive or unnecessary files (like `.env`, `.git`, or `node_modules`) from being added to the final image. +> +> To learn more, visit the [.dockerignore reference](/reference/dockerfile.md#dockerignore-file). + +Create a file named `.dockerignore` with the following contents: + +```dockerignore +# Dependencies (installed inside the image, never copy from host) +node_modules/ +.pnp/ +.pnp.js +.pnpm-store/ + +# Next.js build output (generated during the image build) +.next/ +out/ +dist/ +build/ +.vercel/ + +# Testing (not needed in the production image) +coverage/ +.nyc_output/ +__tests__/ +__mocks__/ +jest/ +cypress/ +playwright-report/ +test-results/ +.vitest/ + +# Environment files (avoid leaking secrets into the build context) +.env +.env* +.env.local +.env.development.local +.env.test.local +.env.production.local + +# Debug and log files +npm-debug.log* +yarn-debug.log* +yarn-error.log* +pnpm-debug.log* +lerna-debug.log* +*.log + +# IDE and editor files +.vscode/ +.idea/ +.cursor/ +.cursorrules +.copilot/ +*.swp +*.swo +*~ + +# Git +.git/ +.gitignore +.gitattributes + +# Docker files (reduce build context; not needed inside the image) +Dockerfile* +.dockerignore +docker-compose*.yml +compose*.yaml + +# Documentation (not needed in the image) +*.md +docs/ + +# CI/CD (not needed in the image) +.github/ +.gitlab-ci.yml +.travis.yml +.circleci/ +Jenkinsfile + +# TypeScript and build metadata +*.tsbuildinfo + +# Cache and temporary directories +.cache/ +.parcel-cache/ +.eslintcache +.stylelintcache +.turbo/ +.tmp/ +.temp/ + +# Sensitive or dev-only config (optional; omit if your build needs these) +.pem +.editorconfig +.prettierrc* +.eslintrc* +.stylelintrc* +.babelrc* +*.iml + +# OS-specific files +.DS_Store +._* +.Spotlight-V100 +.Trashes +ehthumbs.db +Thumbs.db +Desktop.ini +``` + +#### Step 4: Build the Next.js application image + +With your custom configuration in place, you're now ready to build the Docker image. Use the Dockerfile you created in Step 1 (standalone or export). + +The setup includes: + +- Multi-stage builds for optimized image size +- Standalone: Node.js server on port 3000; Export: Nginx serving static files on port 8080 +- Non-root user for enhanced security +- Proper file permissions and ownership + +After completing the previous steps, your project directory should contain at least the following files (export also requires `nginx.conf`): + +```text +├── docker-nextjs-sample/ +│ ├── Dockerfile +│ ├── .dockerignore +│ ├── compose.yaml +│ └── next.config.ts +``` + +Now that your Dockerfile is configured, you can build the Docker image for your Next.js application. + +> [!NOTE] +> The `docker build` command packages your application into an image using the instructions in the Dockerfile. It includes all necessary files from the current directory (called the [build context](/build/concepts/context/#what-is-a-build-context)). + +Run the following command from the root of your project: + +```console +$ docker build --tag nextjs-sample . +``` + +What this command does: +- Uses the Dockerfile in the current directory (.) +- Packages the application and its dependencies into a Docker image +- Tags the image as nextjs-sample so you can reference it later + + +#### Step 5: View local images + +After building your Docker image, you can check which images are available on your local machine using either the Docker CLI or [Docker Desktop](/manuals/desktop/use-desktop/images.md). Since you're already working in the terminal, let's use the Docker CLI. + +To list all locally available Docker images, run the following command: + +```console +$ docker images +``` + +Example Output: + +```shell +REPOSITORY TAG IMAGE ID CREATED SIZE +nextjs-sample latest 8c5fc80f098e 14 seconds ago 130MB +``` + +This output provides key details about your images: + +- Repository – The name assigned to the image. +- Tag – A version label that helps identify different builds (e.g., latest). +- Image ID – A unique identifier for the image. +- Created – The timestamp indicating when the image was built. +- Size – The total disk space used by the image. + +If the build was successful, you should see `nextjs-sample` image listed. + +--- + +### Run the containerized application + +In the previous step, you created a Dockerfile for your Next.js application and built a Docker image using the docker build command. Now it's time to run that image in a container and verify that your application works as expected. + +Run the following command in a terminal. Use the port that matches your setup: standalone uses port 3000, export uses port 8080. + +```console +$ docker run -p 3000:3000 nextjs-sample +``` + +For export output, use port 8080 instead: + +```console +$ docker run -p 8080:8080 nextjs-sample +``` + +Open a browser and view the application: [http://localhost:3000](http://localhost:3000) for standalone or [http://localhost:8080](http://localhost:8080) for export. You should see your Next.js web application. + +Press `ctrl+c` in the terminal to stop your application. + +#### Run the application in the background + +You can run the application detached from the terminal by adding the `-d` option and `--name` to give the container a name so you can stop it later: + +```console +$ docker run -d -p 3000:3000 --name nextjs-app nextjs-sample +``` + +For export output, use port 8080: + +```console +$ docker run -d -p 8080:8080 --name nextjs-app nextjs-sample +``` + +Open a browser and view the application: [http://localhost:3000](http://localhost:3000) for standalone or [http://localhost:8080](http://localhost:8080) for export. You should see your web application. + +To confirm that the container is running, use the `docker ps` command: + +```console +$ docker ps +``` + +This will list all active containers along with their ports, names, and status. Look for a container exposing port 3000 (standalone) or 8080 (export). + +Example Output: + +```shell +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +f49b74736a9d nextjs-sample "node server.js" About a minute ago Up About a minute 0.0.0.0:3000->3000/tcp nextjs-app +``` + +To stop the application, run: + +```console +$ docker stop nextjs-app +``` + +> [!NOTE] +> For more information about running containers, see the [`docker run` CLI reference](/reference/cli/docker/container/run/) and the [`docker stop` CLI reference](/reference/cli/docker/container/stop/). + +--- + +### Summary + +In this guide, you learned how to containerize, build, and run a Next.js application using Docker. By following best practices, you created a secure, optimized, and production-ready setup. + +What you accomplished: +- Configured Next.js for either standalone output (Node.js server) or export output (static files with Nginx). +- Added a multi-stage Dockerfile for your chosen approach: standalone (port 3000) or export (port 8080, with `nginx.conf`). +- Created a `.dockerignore` file to exclude unnecessary files and keep the image clean and efficient. +- Built your Docker image using `docker build`. +- Ran the container using `docker run` with the image name `nextjs-sample`, both in the foreground and in detached mode. +- Verified that the app was running by visiting [http://localhost:3000](http://localhost:3000) (standalone) or [http://localhost:8080](http://localhost:8080) (export). +- Learned how to stop the containerized application using `docker stop nextjs-app`. + +You now have a fully containerized Next.js application, running in a Docker container, and ready for deployment across any environment with confidence and consistency. + +--- + +### Related resources + +Explore official references and best practices to sharpen your Docker workflow: + +- [Multi-stage builds](/build/building/multi-stage/) – Learn how to separate build and runtime stages. +- [Best practices for writing Dockerfiles](/develop/develop-images/dockerfile_best-practices/) – Write efficient, maintainable, and secure Dockerfiles. +- [Build context in Docker](/build/concepts/context/) – Learn how context affects image builds. +- [Next.js output configuration](https://nextjs.org/docs/app/api-reference/config/next-config-js/output) – Learn about Next.js production optimization (standalone and export). +- [Next.js with Docker (standalone)](https://github.com/vercel/next.js/tree/canary/examples/with-docker) – Official Next.js example: standalone output with Node.js. +- [Next.js with Docker (export)](https://github.com/vercel/next.js/tree/canary/examples/with-docker-export-output) – Official Next.js example: static export with Nginx or serve. +- [`docker build` CLI reference](/reference/cli/docker/image/build/) – Build Docker images from a Dockerfile. +- [`docker images` CLI reference](/reference/cli/docker/image/ls/) – Manage and inspect local Docker images. +- [`docker run` CLI reference](/reference/cli/docker/container/run/) – Run a command in a new container. +- [`docker stop` CLI reference](/reference/cli/docker/container/stop/) – Stop one or more running containers. + +--- + +### Next steps + +With your Next.js application now containerized, you're ready to move on to the next step. + +In the next section, you'll learn how to develop your application using Docker containers, enabling a consistent, isolated, and reproducible development environment across any machine. + +## Use containers for Next.js development + +### Prerequisites + +Complete [Containerize Next.js application](containerize.md). + +--- + +### Overview + +In this section, you'll learn how to set up both production and development environments for your containerized Next.js application using Docker Compose. This setup allows you to run a production build using the standalone server and to develop efficiently inside containers using Next.js's built-in hot reloading with Compose Watch. + +You'll learn how to: +- Configure separate containers for production and development +- Enable automatic file syncing using Compose Watch in development +- Debug and live-preview your changes in real-time without manual rebuilds + +--- + +### Automatically update services (development mode) + +Use Compose Watch to automatically sync source file changes into your +containerized development environment. This automatically syncs file changes +without needing to restart or rebuild containers manually. + +### Step 1: Create a development Dockerfile + +Create a file named `Dockerfile.dev` in your project root with the following content (matching the [sample project](https://github.com/kristiyan-velkov/docker-nextjs-sample)): + +```dockerfile +# ============================================ +# Development Dockerfile for Next.js +# ============================================ +ARG NODE_VERSION=24.14.0-slim + +FROM node:${NODE_VERSION} AS dev + +WORKDIR /app + +COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml* .npmrc* ./ + +RUN --mount=type=cache,target=/root/.npm \ + --mount=type=cache,target=/usr/local/share/.cache/yarn \ + --mount=type=cache,target=/root/.local/share/pnpm/store \ + if [ -f package-lock.json ]; then \ + npm ci --no-audit --no-fund; \ + elif [ -f yarn.lock ]; then \ + corepack enable yarn && yarn install --frozen-lockfile --production=false; \ + elif [ -f pnpm-lock.yaml ]; then \ + corepack enable pnpm && pnpm install --frozen-lockfile; \ + else \ + echo "No lockfile found." && exit 1; \ + fi + +COPY . . + +ENV WATCHPACK_POLLING=true +ENV HOSTNAME="0.0.0.0" + +RUN chown -R node:node /app +USER node + +EXPOSE 3000 + +CMD ["sh", "-c", "if [ -f package-lock.json ]; then npm run dev; elif [ -f yarn.lock ]; then yarn dev; elif [ -f pnpm-lock.yaml ]; then pnpm dev; else npm run dev; fi"] +``` + +This file sets up a development environment for your Next.js app with hot module replacement and supports npm, yarn, and pnpm. + + +#### Step 2: Update your `compose.yaml` file + +Open your `compose.yaml` file and define two services: one for production (`nextjs-prod-standalone`) and one for development (`nextjs-dev`). This matches the [sample project](https://github.com/kristiyan-velkov/docker-nextjs-sample) structure. + +Here's an example configuration for a Next.js application: + +```yaml +services: + nextjs-prod-standalone: + build: + context: . + dockerfile: Dockerfile + image: nextjs-sample:prod + container_name: nextjs-sample-prod + ports: + - "3000:3000" + + nextjs-dev: + build: + context: . + dockerfile: Dockerfile.dev + image: nextjs-sample:dev + container_name: nextjs-sample-dev + ports: + - "3000:3000" + environment: + - WATCHPACK_POLLING=true + develop: + watch: + - action: sync + path: . + target: /app + ignore: + - node_modules/ + - .next/ + - action: rebuild + path: package.json +``` + +- The `nextjs-prod-standalone` service builds and runs your production Next.js app using the standalone output. +- The `nextjs-dev` service runs your Next.js development server with hot module replacement. +- `watch` triggers file sync with Compose Watch. +- `WATCHPACK_POLLING=true` ensures file changes are detected properly inside Docker. +- The `rebuild` action for `package.json` ensures dependencies are reinstalled when the file changes. + +> [!NOTE] +> For more details, see the official guide: [Use Compose Watch](/manuals/compose/how-tos/file-watch.md). + +#### Step 3: Configure Next.js for Docker development + +Next.js works well inside Docker containers out of the box, but there are a few configurations that can improve the development experience. + +The `next.config.ts` file you created during containerization already includes the `output: "standalone"` option for production. For development, Next.js automatically uses its built-in development server with hot reloading enabled. + +> [!NOTE] +> The Next.js development server automatically: +> - Enables Hot Module Replacement (HMR) for instant updates +> - Watches for file changes and recompiles automatically +> - Provides detailed error messages in the browser +> +> The `WATCHPACK_POLLING=true` environment variable in the compose file ensures file watching works correctly inside Docker containers. + + +After completing the previous steps, your project directory should now contain the following files: + +```text +├── docker-nextjs-sample/ +│ ├── Dockerfile +│ ├── Dockerfile.dev +│ ├── .dockerignore +│ ├── compose.yaml +│ └── next.config.ts +``` + +#### Step 4: Start Compose Watch + +Run the following command from your project root to start your container in watch mode: + +```console +$ docker compose watch nextjs-dev +``` + +#### Step 5: Test Compose Watch with Next.js + +To verify that Compose Watch is working correctly: + +1. Open the `app/page.tsx` file in your text editor (or `src/app/page.tsx` if your project uses a `src` directory). + +2. Locate the main content area and find a text element to modify. + +3. Make a visible change, for example, update a heading: + + ```tsx +

    Hello from Docker Compose Watch!

    + ``` + +4. Save the file. + +5. Open your browser at [http://localhost:3000](http://localhost:3000). + +You should see the updated text appear instantly, without needing to rebuild the container manually. This confirms that file watching and automatic synchronization are working as expected. + +--- + +### Summary + +In this section, you set up a complete development and production workflow for your Next.js application using Docker and Docker Compose. + +Here's what you achieved: +- Created a `Dockerfile.dev` to streamline local development with hot reloading +- Defined separate `nextjs-dev` and `nextjs-prod-standalone` services in your `compose.yaml` file +- Enabled real-time file syncing using Compose Watch for a smoother development experience +- Verified that live updates work seamlessly by modifying and previewing a component + +With this setup, you can build, run, and iterate on your Next.js app +entirely within containers across environments. + +--- + +### Related resources + +Deepen your knowledge and improve your containerized development workflow with these guides: + +- [Using Compose Watch](/manuals/compose/how-tos/file-watch.md) – Automatically sync source changes during development +- [Multi-stage builds](/manuals/build/building/multi-stage.md) – Create efficient, production-ready Docker images +- [Dockerfile best practices](/build/building/best-practices/) – Write clean, secure, and optimized Dockerfiles. +- [Compose file reference](/compose/compose-file/) – Learn the full syntax and options available for configuring services in `compose.yaml`. +- [Docker volumes](/storage/volumes/) – Persist and manage data between container runs + +### Next steps + +In the next section, you'll learn how to run unit tests for your Next.js application inside Docker containers. This ensures consistent testing across all environments and removes dependencies on local machine setup. + +## Run Next.js tests in a container + +### Prerequisites + +Complete all the previous sections of this guide, starting with [Containerize Next.js application](containerize.md). + +### Overview + +Testing is a critical part of the development process. In this section, you'll learn how to: + +- Run unit tests using Vitest (or Jest) inside a Docker container. +- Run lint (e.g. ESLint) inside a Docker container. +- Use Docker Compose to run tests and lint in an isolated, reproducible environment. + +The [sample project](https://github.com/kristiyan-velkov/docker-nextjs-sample) uses [Vitest](https://vitest.dev/) with [Testing Library](https://testing-library.com/) for component testing. You can use the same setup or follow the alternative Jest configuration later. + +--- + +### Run tests during development + +The [sample project](https://github.com/kristiyan-velkov/docker-nextjs-sample) already includes lint (ESLint) and sample tests (Vitest, `app/page.test.tsx`) in place. If you're using the sample app, you can skip to **Step 3: Update compose.yaml** and run tests or lint with the commands below. If you're using your own project, follow the install and configuration steps to add the packages and scripts. + +The sample includes a test file at: + +```text +app/page.test.tsx +``` + +This file uses Vitest and React Testing Library to verify the behavior of page components. + +#### Step 1: Install Vitest and React Testing Library (custom projects) + +If you're using a custom project and haven't already added the necessary testing tools, install them by running: + +```console +$ npm install --save-dev vitest @vitejs/plugin-react @testing-library/react @testing-library/dom jsdom +``` + +Then, update the scripts section of your `package.json` file to include: + +```json +"scripts": { + "test": "vitest", + "test:run": "vitest run" +} +``` + +For lint, add a `lint` script (and optionally `lint:fix`). For example, with [ESLint](https://eslint.org/): + +```json +"scripts": { + "test": "vitest", + "test:run": "vitest run", + "lint": "eslint .", + "lint:fix": "eslint . --fix" +} +``` + +The sample project uses `eslint` and `eslint-config-next` for Next.js. Install them in a custom project with: + +```console +$ npm install --save-dev eslint eslint-config-next @eslint/eslintrc +``` + +Create an ESLint config file (e.g. `eslint.config.cjs`) in your project root with Next.js rules and global ignores: + +```js +const { defineConfig, globalIgnores } = require("eslint/config"); +const { FlatCompat } = require("@eslint/eslintrc"); + +const compat = new FlatCompat({ baseDirectory: __dirname }); + +module.exports = defineConfig([ + ...compat.extends( + "eslint-config-next/core-web-vitals", + "eslint-config-next/typescript" + ), + globalIgnores([ + ".next/**", + "out/**", + "build/**", + "next-env.d.ts", + "node_modules/**", + "eslint.config.cjs", + ]), +]); +``` + +--- + +#### Step 2: Configure Vitest (custom projects) + +If you're using a custom project, create a `vitest.config.ts` file in your project root (matching the [sample project](https://github.com/kristiyan-velkov/docker-nextjs-sample)): + +```ts +import { defineConfig } from "vitest/config"; +import react from "@vitejs/plugin-react"; + +export default defineConfig({ + plugins: [react()], + test: { + environment: "jsdom", + setupFiles: "./vitest.setup.ts", + globals: true, + }, +}); +``` + +Create a `vitest.setup.ts` file in your project root: + +```ts +import "@testing-library/jest-dom/vitest"; +``` + +> [!NOTE] +> Vitest works well with Next.js and provides fast execution and ESM support. For more details, see the [Next.js testing documentation](https://nextjs.org/docs/app/building-your-application/testing) and [Vitest docs](https://vitest.dev/). + +#### Step 3: Update compose.yaml + +Add `nextjs-test` and `nextjs-lint` services to your `compose.yaml` file. In the sample project these services use the `tools` profile so they don't start with a normal `docker compose up`. Both reuse `Dockerfile.dev` and run the test or lint command: + +```yaml +services: + nextjs-prod-standalone: + build: + context: . + dockerfile: Dockerfile + image: nextjs-sample:prod + container_name: nextjs-sample-prod + ports: + - "3000:3000" + + nextjs-dev: + build: + context: . + dockerfile: Dockerfile.dev + image: nextjs-sample:dev + container_name: nextjs-sample-dev + ports: + - "3000:3000" + environment: + - WATCHPACK_POLLING=true + develop: + watch: + - action: sync + path: . + target: /app + ignore: + - node_modules/ + - .next/ + - action: rebuild + path: package.json + + nextjs-test: + build: + context: . + dockerfile: Dockerfile.dev + image: nextjs-sample:dev + container_name: nextjs-sample-test + command: + [ + "sh", + "-c", + "if [ -f package-lock.json ]; then npm run test:run 2>/dev/null || npm run test -- --run; elif [ -f yarn.lock ]; then yarn test:run 2>/dev/null || yarn test --run; elif [ -f pnpm-lock.yaml ]; then pnpm run test:run; else npm run test -- --run; fi", + ] + profiles: + - tools + + nextjs-lint: + build: + context: . + dockerfile: Dockerfile.dev + image: nextjs-sample:dev + container_name: nextjs-sample-lint + command: + [ + "sh", + "-c", + "if [ -f package-lock.json ]; then npm run lint; elif [ -f yarn.lock ]; then yarn lint; elif [ -f pnpm-lock.yaml ]; then pnpm lint; else npm run lint; fi", + ] + profiles: + - tools +``` + +The `nextjs-test` and `nextjs-lint` services reuse the same `Dockerfile.dev` used for [development](develop.md) and override the default command to run tests or lint. The `profiles: [tools]` means these services only run when you use the `--profile tools` option. + +After completing the previous steps, your project directory should contain: + +```text +├── docker-nextjs-sample/ +│ ├── Dockerfile +│ ├── Dockerfile.dev +│ ├── .dockerignore +│ ├── compose.yaml +│ ├── vitest.config.ts +│ ├── vitest.setup.ts +│ └── next.config.ts +``` + +#### Step 4: Run the tests + +To execute your test suite inside the container, run from your project root: + +```console +$ docker compose --profile tools run --rm nextjs-test +``` + +This command will: +- Start the `nextjs-test` service (because of `--profile tools`). +- Run your test script (`test:run` or `test -- --run`) in the same environment as development. +- Remove the container after the tests complete ([`docker compose run --rm`](/reference/cli/docker/compose/run/)). + +> [!NOTE] +> For more information about Compose commands and profiles, see the [Compose CLI reference](/reference/cli/docker/compose/). + +#### Step 5: Run lint in the container + +To run your linter (e.g. ESLint) inside the container, use the `nextjs-lint` service with the same `tools` profile: + +```console +$ docker compose --profile tools run --rm nextjs-lint +``` + +This command will: +- Start the `nextjs-lint` service (because of `--profile tools`). +- Run your lint script (`npm run lint`, `yarn lint`, or `pnpm lint` depending on your lockfile) in the same environment as development. +- Remove the container after lint completes. + +Ensure your `package.json` includes a `lint` script. The sample project already has `"lint": "eslint ."` and `"lint:fix": "eslint . --fix"`; for a custom project, add the same and install `eslint` and `eslint-config-next` if needed. + +--- + +### Summary + +In this section, you learned how to run unit tests for your Next.js application inside a Docker container using Vitest and Docker Compose. + +What you accomplished: +- Installed and configured Vitest and React Testing Library for testing Next.js components. +- Created `nextjs-test` and `nextjs-lint` services in `compose.yaml` (with `tools` profile) to isolate test and lint execution. +- Reused the development `Dockerfile.dev` to ensure consistency between dev, test, and lint environments. +- Ran tests inside the container using `docker compose --profile tools run --rm nextjs-test`. +- Ran lint inside the container using `docker compose --profile tools run --rm nextjs-lint`. +- Ensured reliable, repeatable testing and linting across environments without relying on local machine setup. + +--- + +### Related resources + +Explore official references and best practices to sharpen your Docker testing workflow: + +- [Dockerfile reference](/reference/dockerfile/) – Understand all Dockerfile instructions and syntax. +- [Best practices for writing Dockerfiles](/develop/develop-images/dockerfile_best-practices/) – Write efficient, maintainable, and secure Dockerfiles. +- [Compose file reference](/compose/compose-file/) – Learn the full syntax and options available for configuring services in `compose.yaml`. +- [`docker compose run` CLI reference](/reference/cli/docker/compose/run/) – Run one-off commands in a service container. +- [Next.js Testing Documentation](https://nextjs.org/docs/app/building-your-application/testing) – Official Next.js testing guide. +--- + +### Next steps + +Next, you'll learn how to set up a CI/CD pipeline using GitHub Actions to automatically build and test your Next.js application in a containerized environment. This ensures your code is validated on every push or pull request, maintaining consistency and reliability across your development workflow. + +## Automate your builds with GitHub Actions + +### Prerequisites + +Complete all the previous sections of this guide, starting with [Containerize Next.js application](containerize.md). + +You must also have: +- A [GitHub](https://github.com/signup) account. +- A verified [Docker Hub](https://hub.docker.com/signup) account. + +--- + +### Overview + +In this section, you'll set up a CI/CD pipeline using [GitHub Actions](https://docs.github.com/en/actions) to automatically: + +- Build your Next.js application inside a Docker container. +- Run tests in a consistent environment. +- Push the production-ready image to [Docker Hub](https://hub.docker.com). + +--- + +### Integrate GitHub and Docker Hub + +To enable GitHub Actions to build and push Docker images, you'll securely +store your Docker Hub credentials in your new GitHub repository. + +#### Step 1: Connect your GitHub repository to Docker Hub + +1. Create a Personal Access Token (PAT) from [Docker Hub](https://hub.docker.com) + 1. Go to your **Docker Hub account → Account Settings → Security**. + 2. Generate a new Access Token with **Read/Write** permissions. + 3. Name it something like `nextjs-sample`. + 4. Copy and save the token — you'll need it in Step 4. + +2. Create a repository in [Docker Hub](https://hub.docker.com/repositories/) + 1. Go to your **Docker Hub account → Create a repository**. + 2. For the Repository Name, use something descriptive — for example: `nextjs-sample`. + 3. Once created, copy and save the repository name — you'll need it in Step 4. + +3. Create a new [GitHub repository](https://github.com/new) for your Next.js project + +4. Add Docker Hub credentials as GitHub repository secrets + + In your newly created GitHub repository: + + 1. Navigate to: + **Settings → Secrets and variables → Actions → New repository secret**. + + 2. Add the following secrets: + + | Name | Value | + |-------------------|--------------------------------| + | `DOCKER_USERNAME` | Your Docker Hub username | + | `DOCKERHUB_TOKEN` | Your Docker Hub access token (created in Step 1) | + | `DOCKERHUB_PROJECT_NAME` | Your Docker Project Name (created in Step 2) | + + These secrets let GitHub Actions authenticate securely with Docker Hub + during automated workflows. + +5. Connect Your Local Project to GitHub + + Link your local project to the GitHub repository you just created by running the following command from your project root: + + ```console + $ git remote set-url origin https://github.com/{your-username}/{your-repository-name}.git + ``` + + >[!IMPORTANT] + >Replace `{your-username}` and `{your-repository}` with your actual GitHub username and repository name. + + To confirm that your local project is correctly connected to the remote GitHub repository, run: + + ```console + $ git remote -v + ``` + + You should see output similar to: + + ```console + origin https://github.com/{your-username}/{your-repository-name}.git (fetch) + origin https://github.com/{your-username}/{your-repository-name}.git (push) + ``` + + This confirms that your local repository is properly linked and ready to push your source code to GitHub. + +6. Push Your Source Code to GitHub + + Follow these steps to commit and push your local project to your GitHub repository: + + 1. Stage all files for commit. + + ```console + $ git add -A + ``` + This command stages all changes — including new, modified, and deleted files — preparing them for commit. + + + 2. Commit your changes. + + ```console + $ git commit -m "Initial commit" + ``` + This command creates a commit that snapshots the staged changes with a descriptive message. + + 3. Push the code to the `main` branch. + + ```console + $ git push -u origin main + ``` + This command pushes your local commits to the `main` branch of the remote GitHub repository and sets the upstream branch. + +Once completed, your code will be available on GitHub, and any GitHub Actions workflow you've configured will run automatically. + +> [!NOTE] +> Learn more about the Git commands used in this step: +> - [Git add](https://git-scm.com/docs/git-add) – Stage changes (new, modified, deleted) for commit +> - [Git commit](https://git-scm.com/docs/git-commit) – Save a snapshot of your staged changes +> - [Git push](https://git-scm.com/docs/git-push) – Upload local commits to your GitHub repository +> - [Git remote](https://git-scm.com/docs/git-remote) – View and manage remote repository URLs + +--- + +#### Step 2: Set up the workflow + +Now you'll create a GitHub Actions workflow that builds your Docker image, runs tests, and pushes the image to Docker Hub. + +1. Go to your repository on GitHub and select the **Actions** tab in the top menu. + +2. Select **Set up a workflow yourself** when prompted. + + This opens an inline editor to create a new workflow file. By default, it will be saved to: + `.github/workflows/main.yml` + + +3. Add the following workflow configuration to the new file: + +```yaml +# CI/CD – Next.js Application with Docker +# Builds the app, runs tests in a container, and pushes the production image to Docker Hub. + +name: CI/CD – Next.js Application with Docker + +on: + push: + branches: [main] + pull_request: + branches: [main] + types: [opened, synchronize, reopened] + +jobs: + build-test-push: + name: Build, Test and Push Docker Image + runs-on: ubuntu-latest + + steps: + # 1. Checkout source code + - name: Checkout source code + uses: actions/checkout@v5 + with: + fetch-depth: 0 + + # 2. Set up Docker Buildx + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@v4 + + # 3. Cache Docker layers + - name: Cache Docker layers + uses: actions/cache@v5 + with: + path: /tmp/.buildx-cache + key: ${{ runner.os }}-buildx-${{ github.sha }} + restore-keys: ${{ runner.os }}-buildx- + + # 4. Cache pnpm dependencies + - name: Cache pnpm dependencies + uses: actions/cache@v4 + with: + path: ~/.local/share/pnpm/store + key: ${{ runner.os }}-pnpm-${{ hashFiles('**/pnpm-lock.yaml') }} + restore-keys: ${{ runner.os }}-pnpm- + + # 5. Extract metadata + - name: Extract metadata + id: meta + run: | + echo "REPO_NAME=${GITHUB_REPOSITORY##*/}" >> "$GITHUB_OUTPUT" + echo "SHORT_SHA=${GITHUB_SHA::7}" >> "$GITHUB_OUTPUT" + + # 6. Build dev Docker image (for running tests) + - name: Build Docker image for tests + uses: docker/build-push-action@v6 + with: + context: . + file: Dockerfile.dev + tags: ${{ steps.meta.outputs.REPO_NAME }}-dev:latest + load: true + cache-from: type=local,src=/tmp/.buildx-cache + cache-to: type=local,dest=/tmp/.buildx-cache,mode=max + + # 7. Run Vitest tests inside the container + # Use same package-manager detection as Dockerfile (no corepack at runtime; node user can't write to /usr/local/bin) + - name: Run tests + run: | + docker run --rm \ + --workdir /app \ + --entrypoint "" \ + -e CI=true \ + ${{ steps.meta.outputs.REPO_NAME }}-dev:latest \ + sh -c "if [ -f package-lock.json ]; then npm run test:run; elif [ -f yarn.lock ]; then yarn test:run; elif [ -f pnpm-lock.yaml ]; then pnpm run test:run; else npm run test:run; fi" + env: + CI: true + NODE_ENV: test + timeout-minutes: 10 + + # 8. Log in to Docker Hub (only needed for push) + - name: Log in to Docker Hub + if: github.event_name == 'push' && github.ref == 'refs/heads/main' + uses: docker/login-action@v3 + with: + username: ${{ secrets.DOCKER_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + + # 9. Build and push production image (only on push to main) + - name: Build and push production image + if: github.event_name == 'push' && github.ref == 'refs/heads/main' + uses: docker/build-push-action@v6 + with: + context: . + file: Dockerfile + push: true + platforms: linux/amd64,linux/arm64 + tags: | + ${{ secrets.DOCKER_USERNAME }}/${{ secrets.DOCKERHUB_PROJECT_NAME }}:latest + ${{ secrets.DOCKER_USERNAME }}/${{ secrets.DOCKERHUB_PROJECT_NAME }}:${{ steps.meta.outputs.SHORT_SHA }} + cache-from: type=local,src=/tmp/.buildx-cache + cache-to: type=local,dest=/tmp/.buildx-cache,mode=max + +``` + +This workflow performs the following tasks for your Next.js application: +- Triggers on every `push` or `pull request` targeting the `main` branch. +- Builds a development Docker image using `Dockerfile.dev`, optimized for testing. +- Executes unit tests using Jest inside a clean, containerized environment to ensure consistency. +- Halts the workflow immediately if any test fails — enforcing code quality. +- Caches both Docker build layers and npm dependencies for faster CI runs. +- Authenticates securely with Docker Hub using GitHub repository secrets. +- Builds a production-ready image using the `Dockerfile`. +- Tags and pushes the final image to Docker Hub with both `latest` and short SHA tags for traceability. + +> [!NOTE] +> For more information about `docker/build-push-action`, refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). + +--- + +#### Step 3: Run the workflow + +After you've added your workflow file, it's time to trigger and observe the CI/CD process in action. + +1. Commit and push your workflow file + + Select "Commit changes…" in the GitHub editor. + + - This push will automatically trigger the GitHub Actions pipeline. + +2. Monitor the workflow execution + + 1. Go to the Actions tab in your GitHub repository. + 2. Select the workflow run to follow each step: `build`, `test`, and (if successful) `push`. + +3. Verify the Docker image on Docker Hub + + - After a successful workflow run, visit your [Docker Hub repositories](https://hub.docker.com/repositories). + - You should see a new image under your repository with: + - Repository name: `${your-repository-name}` + - Tags include: + - `latest` – represents the most recent successful build; ideal for quick testing or deployment. + - `` – a unique identifier based on the commit hash, useful for version tracking, rollbacks, and traceability. + +> [!TIP] Protect your main branch +> To maintain code quality and prevent accidental direct pushes, enable branch protection rules: +> - Navigate to your **GitHub repo → Settings → Branches**. +> - Under Branch protection rules, select **Add rule**. +> - Specify `main` as the branch name. +> - Enable options like: +> - *Require a pull request before merging*. +> - *Require status checks to pass before merging*. +> +> This ensures that only tested and reviewed code is merged into `main` branch. +--- + +### Summary + +In this section, you set up a complete CI/CD pipeline for your containerized Next.js application using GitHub Actions. + +Here's what you accomplished: + +- Created a new GitHub repository specifically for your project. +- Generated a secure Docker Hub access token and added it to GitHub as a secret. +- Defined a GitHub Actions workflow to: + - Build your application inside a Docker container. + - Run tests in a consistent, containerized environment. + - Push a production-ready image to Docker Hub if tests pass. +- Triggered and verified the workflow execution through GitHub Actions. +- Confirmed that your image was successfully published to Docker Hub. + +With this setup, your Next.js application is now ready for automated testing and deployment across environments — increasing confidence, consistency, and team productivity. + +--- + +### Related resources + +Deepen your understanding of automation and best practices for containerized apps: + +- [Introduction to GitHub Actions](/guides/gha.md) – Learn how GitHub Actions automate your workflows +- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) – Set up container builds with GitHub Actions +- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) – Full reference for writing GitHub workflows +- [Compose file reference](/compose/compose-file/) – Full configuration reference for `compose.yaml` +- [Best practices for writing Dockerfiles](/develop/develop-images/dockerfile_best-practices/) – Optimize your image for performance and security + +--- + +### Next steps + +Next, learn how you can locally test and debug your Next.js workloads on Kubernetes before deploying. This helps you ensure your application behaves as expected in a production-like environment, reducing surprises during deployment. + +## Test your Next.js deployment + +### Prerequisites + +Before you begin, make sure you've completed the following: +- Complete all the previous sections of this guide, starting with [Containerize Next.js application](containerize.md). +- [Enable Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. + +> **New to Kubernetes?** +> Visit the [Kubernetes basics tutorial](https://kubernetes.io/docs/tutorials/kubernetes-basics/) to get familiar with how clusters, pods, deployments, and services work. + +--- + +### Overview + +This section guides you through deploying your containerized Next.js application locally using [Docker Desktop's built-in Kubernetes](/desktop/kubernetes/). Running your app in a local Kubernetes cluster allows you to closely simulate a real production environment, enabling you to test, validate, and debug your workloads with confidence before promoting them to staging or production. + +--- + +### Create a Kubernetes YAML file + +Follow these steps to define your deployment configuration: + +1. In the root of your project, create a new file named: nextjs-sample-kubernetes.yaml + +2. Open the file in your IDE or preferred text editor. + +3. Add the following configuration, and be sure to replace `{DOCKER_USERNAME}` and `{DOCKERHUB_PROJECT_NAME}` with your actual Docker Hub username and repository name from the previous [Automate your builds with GitHub Actions](configure-github-actions.md). + + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: nextjs-sample + namespace: default +spec: + replicas: 1 + selector: + matchLabels: + app: nextjs-sample + template: + metadata: + labels: + app: nextjs-sample + spec: + containers: + - name: nextjs-container + image: {DOCKER_USERNAME}/{DOCKERHUB_PROJECT_NAME}:latest + imagePullPolicy: Always + ports: + - containerPort: 3000 + env: + - name: NODE_ENV + value: "production" + - name: HOSTNAME + value: "0.0.0.0" +--- +apiVersion: v1 +kind: Service +metadata: + name: nextjs-sample-service + namespace: default +spec: + type: NodePort + selector: + app: nextjs-sample + ports: + - port: 3000 + targetPort: 3000 + nodePort: 30001 +``` + +This manifest defines two key Kubernetes resources, separated by `---`: + +- Deployment + Deploys a single replica of your Next.js application inside a pod. The pod uses the Docker image built and pushed by your GitHub Actions CI/CD workflow + (refer to [Automate your builds with GitHub Actions](configure-github-actions.md)). + The container listens on port `3000`, which is the default port for Next.js applications. + +- Service (NodePort) + Exposes the deployed pod to your local machine. + It forwards traffic from port `30001` on your host to port `3000` inside the container. + This lets you access the application in your browser at [http://localhost:30001](http://localhost:30001). + +> [!NOTE] +> To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). + +--- + +### Deploy and check your application + +Follow these steps to deploy your containerized Next.js app into a local Kubernetes cluster and verify that it's running correctly. + +#### Step 1. Apply the Kubernetes configuration + +In your terminal, navigate to the directory where your `nextjs-sample-kubernetes.yaml` file is located, then deploy the resources using: + +```console + $ kubectl apply -f nextjs-sample-kubernetes.yaml +``` + +If everything is configured properly, you'll see confirmation that both the Deployment and the Service were created: + +```shell + deployment.apps/nextjs-sample created + service/nextjs-sample-service created +``` + +This output means that both the Deployment and the Service were successfully created and are now running inside your local cluster. + +#### Step 2. Check the deployment status + +Run the following command to check the status of your deployment: + +```console + $ kubectl get deployments +``` + +You should see an output similar to: + +```shell + NAME READY UP-TO-DATE AVAILABLE AGE + nextjs-sample 1/1 1 1 14s +``` + +This confirms that your pod is up and running with one replica available. + +#### Step 3. Verify the service exposure + +Check if the NodePort service is exposing your app to your local machine: + +```console +$ kubectl get services +``` + +You should see something like: + +```shell +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +nextjs-sample-service NodePort 10.100.244.65 3000:30001/TCP 1m +``` + +This output confirms that your app is available via NodePort on port 30001. + +#### Step 4. Access your app in the browser + +Open your browser and navigate to [http://localhost:30001](http://localhost:30001). + +You should see your production-ready Next.js Sample application running — served by your local Kubernetes cluster. + +#### Step 5. Clean up Kubernetes resources + +Once you're done testing, you can delete the deployment and service using: + +```console + $ kubectl delete -f nextjs-sample-kubernetes.yaml +``` + +Expected output: + +```shell + deployment.apps "nextjs-sample" deleted + service "nextjs-sample-service" deleted +``` + +This ensures your cluster stays clean and ready for the next deployment. + +--- + +### Summary + +In this section, you learned how to deploy your Next.js application to a local Kubernetes cluster using Docker Desktop. This setup allows you to test and debug your containerized app in a production-like environment before deploying it to the cloud. + +What you accomplished: + +- Created a Kubernetes Deployment and NodePort Service for your Next.js app +- Used `kubectl apply` to deploy the application locally +- Verified the app was running and accessible at `http://localhost:30001` +- Cleaned up your Kubernetes resources after testing + +--- + +### Related resources + +Explore official references and best practices to sharpen your Kubernetes deployment workflow: + +- [Kubernetes documentation](https://kubernetes.io/docs/home/) – Learn about core concepts, workloads, services, and more. +- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) – Use Docker Desktop's built-in Kubernetes support for local testing and development. +- [`kubectl` CLI reference](https://kubernetes.io/docs/reference/kubectl/) – Manage Kubernetes clusters from the command line. +- [Kubernetes Deployment resource](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) – Understand how to manage and scale applications using Deployments. +- [Kubernetes Service resource](https://kubernetes.io/docs/concepts/services-networking/service/) – Learn how to expose your application to internal and external traffic. diff --git a/content/guides/nextjs/configure-github-actions.md b/content/guides/nextjs/configure-github-actions.md deleted file mode 100644 index 2484e5a11b78..000000000000 --- a/content/guides/nextjs/configure-github-actions.md +++ /dev/null @@ -1,332 +0,0 @@ ---- -title: Automate your builds with GitHub Actions -linkTitle: GitHub Actions CI -weight: 60 -keywords: CI/CD, GitHub Actions, Next.js -description: Learn how to configure CI/CD using GitHub Actions for your Next.js application. - ---- - -## Prerequisites - -Complete all the previous sections of this guide, starting with [Containerize Next.js application](containerize.md). - -You must also have: -- A [GitHub](https://github.com/signup) account. -- A verified [Docker Hub](https://hub.docker.com/signup) account. - ---- - -## Overview - -In this section, you'll set up a CI/CD pipeline using [GitHub Actions](https://docs.github.com/en/actions) to automatically: - -- Build your Next.js application inside a Docker container. -- Run tests in a consistent environment. -- Push the production-ready image to [Docker Hub](https://hub.docker.com). - ---- - -## Integrate GitHub and Docker Hub - -To enable GitHub Actions to build and push Docker images, you'll securely -store your Docker Hub credentials in your new GitHub repository. - -### Step 1: Connect your GitHub repository to Docker Hub - -1. Create a Personal Access Token (PAT) from [Docker Hub](https://hub.docker.com) - 1. Go to your **Docker Hub account → Account Settings → Security**. - 2. Generate a new Access Token with **Read/Write** permissions. - 3. Name it something like `nextjs-sample`. - 4. Copy and save the token — you'll need it in Step 4. - -2. Create a repository in [Docker Hub](https://hub.docker.com/repositories/) - 1. Go to your **Docker Hub account → Create a repository**. - 2. For the Repository Name, use something descriptive — for example: `nextjs-sample`. - 3. Once created, copy and save the repository name — you'll need it in Step 4. - -3. Create a new [GitHub repository](https://github.com/new) for your Next.js project - -4. Add Docker Hub credentials as GitHub repository secrets - - In your newly created GitHub repository: - - 1. Navigate to: - **Settings → Secrets and variables → Actions → New repository secret**. - - 2. Add the following secrets: - - | Name | Value | - |-------------------|--------------------------------| - | `DOCKER_USERNAME` | Your Docker Hub username | - | `DOCKERHUB_TOKEN` | Your Docker Hub access token (created in Step 1) | - | `DOCKERHUB_PROJECT_NAME` | Your Docker Project Name (created in Step 2) | - - These secrets let GitHub Actions authenticate securely with Docker Hub - during automated workflows. - -5. Connect Your Local Project to GitHub - - Link your local project to the GitHub repository you just created by running the following command from your project root: - - ```console - $ git remote set-url origin https://github.com/{your-username}/{your-repository-name}.git - ``` - - >[!IMPORTANT] - >Replace `{your-username}` and `{your-repository}` with your actual GitHub username and repository name. - - To confirm that your local project is correctly connected to the remote GitHub repository, run: - - ```console - $ git remote -v - ``` - - You should see output similar to: - - ```console - origin https://github.com/{your-username}/{your-repository-name}.git (fetch) - origin https://github.com/{your-username}/{your-repository-name}.git (push) - ``` - - This confirms that your local repository is properly linked and ready to push your source code to GitHub. - -6. Push Your Source Code to GitHub - - Follow these steps to commit and push your local project to your GitHub repository: - - 1. Stage all files for commit. - - ```console - $ git add -A - ``` - This command stages all changes — including new, modified, and deleted files — preparing them for commit. - - - 2. Commit your changes. - - ```console - $ git commit -m "Initial commit" - ``` - This command creates a commit that snapshots the staged changes with a descriptive message. - - 3. Push the code to the `main` branch. - - ```console - $ git push -u origin main - ``` - This command pushes your local commits to the `main` branch of the remote GitHub repository and sets the upstream branch. - -Once completed, your code will be available on GitHub, and any GitHub Actions workflow you've configured will run automatically. - -> [!NOTE] -> Learn more about the Git commands used in this step: -> - [Git add](https://git-scm.com/docs/git-add) – Stage changes (new, modified, deleted) for commit -> - [Git commit](https://git-scm.com/docs/git-commit) – Save a snapshot of your staged changes -> - [Git push](https://git-scm.com/docs/git-push) – Upload local commits to your GitHub repository -> - [Git remote](https://git-scm.com/docs/git-remote) – View and manage remote repository URLs - ---- - -### Step 2: Set up the workflow - -Now you'll create a GitHub Actions workflow that builds your Docker image, runs tests, and pushes the image to Docker Hub. - -1. Go to your repository on GitHub and select the **Actions** tab in the top menu. - -2. Select **Set up a workflow yourself** when prompted. - - This opens an inline editor to create a new workflow file. By default, it will be saved to: - `.github/workflows/main.yml` - - -3. Add the following workflow configuration to the new file: - -```yaml -# CI/CD – Next.js Application with Docker -# Builds the app, runs tests in a container, and pushes the production image to Docker Hub. - -name: CI/CD – Next.js Application with Docker - -on: - push: - branches: [main] - pull_request: - branches: [main] - types: [opened, synchronize, reopened] - -jobs: - build-test-push: - name: Build, Test and Push Docker Image - runs-on: ubuntu-latest - - steps: - # 1. Checkout source code - - name: Checkout source code - uses: actions/checkout@v5 - with: - fetch-depth: 0 - - # 2. Set up Docker Buildx - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@v4 - - # 3. Cache Docker layers - - name: Cache Docker layers - uses: actions/cache@v5 - with: - path: /tmp/.buildx-cache - key: ${{ runner.os }}-buildx-${{ github.sha }} - restore-keys: ${{ runner.os }}-buildx- - - # 4. Cache pnpm dependencies - - name: Cache pnpm dependencies - uses: actions/cache@v4 - with: - path: ~/.local/share/pnpm/store - key: ${{ runner.os }}-pnpm-${{ hashFiles('**/pnpm-lock.yaml') }} - restore-keys: ${{ runner.os }}-pnpm- - - # 5. Extract metadata - - name: Extract metadata - id: meta - run: | - echo "REPO_NAME=${GITHUB_REPOSITORY##*/}" >> "$GITHUB_OUTPUT" - echo "SHORT_SHA=${GITHUB_SHA::7}" >> "$GITHUB_OUTPUT" - - # 6. Build dev Docker image (for running tests) - - name: Build Docker image for tests - uses: docker/build-push-action@v6 - with: - context: . - file: Dockerfile.dev - tags: ${{ steps.meta.outputs.REPO_NAME }}-dev:latest - load: true - cache-from: type=local,src=/tmp/.buildx-cache - cache-to: type=local,dest=/tmp/.buildx-cache,mode=max - - # 7. Run Vitest tests inside the container - # Use same package-manager detection as Dockerfile (no corepack at runtime; node user can't write to /usr/local/bin) - - name: Run tests - run: | - docker run --rm \ - --workdir /app \ - --entrypoint "" \ - -e CI=true \ - ${{ steps.meta.outputs.REPO_NAME }}-dev:latest \ - sh -c "if [ -f package-lock.json ]; then npm run test:run; elif [ -f yarn.lock ]; then yarn test:run; elif [ -f pnpm-lock.yaml ]; then pnpm run test:run; else npm run test:run; fi" - env: - CI: true - NODE_ENV: test - timeout-minutes: 10 - - # 8. Log in to Docker Hub (only needed for push) - - name: Log in to Docker Hub - if: github.event_name == 'push' && github.ref == 'refs/heads/main' - uses: docker/login-action@v3 - with: - username: ${{ secrets.DOCKER_USERNAME }} - password: ${{ secrets.DOCKERHUB_TOKEN }} - - # 9. Build and push production image (only on push to main) - - name: Build and push production image - if: github.event_name == 'push' && github.ref == 'refs/heads/main' - uses: docker/build-push-action@v6 - with: - context: . - file: Dockerfile - push: true - platforms: linux/amd64,linux/arm64 - tags: | - ${{ secrets.DOCKER_USERNAME }}/${{ secrets.DOCKERHUB_PROJECT_NAME }}:latest - ${{ secrets.DOCKER_USERNAME }}/${{ secrets.DOCKERHUB_PROJECT_NAME }}:${{ steps.meta.outputs.SHORT_SHA }} - cache-from: type=local,src=/tmp/.buildx-cache - cache-to: type=local,dest=/tmp/.buildx-cache,mode=max - -``` - -This workflow performs the following tasks for your Next.js application: -- Triggers on every `push` or `pull request` targeting the `main` branch. -- Builds a development Docker image using `Dockerfile.dev`, optimized for testing. -- Executes unit tests using Jest inside a clean, containerized environment to ensure consistency. -- Halts the workflow immediately if any test fails — enforcing code quality. -- Caches both Docker build layers and npm dependencies for faster CI runs. -- Authenticates securely with Docker Hub using GitHub repository secrets. -- Builds a production-ready image using the `Dockerfile`. -- Tags and pushes the final image to Docker Hub with both `latest` and short SHA tags for traceability. - -> [!NOTE] -> For more information about `docker/build-push-action`, refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). - ---- - -### Step 3: Run the workflow - -After you've added your workflow file, it's time to trigger and observe the CI/CD process in action. - -1. Commit and push your workflow file - - Select "Commit changes…" in the GitHub editor. - - - This push will automatically trigger the GitHub Actions pipeline. - -2. Monitor the workflow execution - - 1. Go to the Actions tab in your GitHub repository. - 2. Select the workflow run to follow each step: `build`, `test`, and (if successful) `push`. - -3. Verify the Docker image on Docker Hub - - - After a successful workflow run, visit your [Docker Hub repositories](https://hub.docker.com/repositories). - - You should see a new image under your repository with: - - Repository name: `${your-repository-name}` - - Tags include: - - `latest` – represents the most recent successful build; ideal for quick testing or deployment. - - `` – a unique identifier based on the commit hash, useful for version tracking, rollbacks, and traceability. - -> [!TIP] Protect your main branch -> To maintain code quality and prevent accidental direct pushes, enable branch protection rules: -> - Navigate to your **GitHub repo → Settings → Branches**. -> - Under Branch protection rules, select **Add rule**. -> - Specify `main` as the branch name. -> - Enable options like: -> - *Require a pull request before merging*. -> - *Require status checks to pass before merging*. -> -> This ensures that only tested and reviewed code is merged into `main` branch. ---- - -## Summary - -In this section, you set up a complete CI/CD pipeline for your containerized Next.js application using GitHub Actions. - -Here's what you accomplished: - -- Created a new GitHub repository specifically for your project. -- Generated a secure Docker Hub access token and added it to GitHub as a secret. -- Defined a GitHub Actions workflow to: - - Build your application inside a Docker container. - - Run tests in a consistent, containerized environment. - - Push a production-ready image to Docker Hub if tests pass. -- Triggered and verified the workflow execution through GitHub Actions. -- Confirmed that your image was successfully published to Docker Hub. - -With this setup, your Next.js application is now ready for automated testing and deployment across environments — increasing confidence, consistency, and team productivity. - ---- - -## Related resources - -Deepen your understanding of automation and best practices for containerized apps: - -- [Introduction to GitHub Actions](/guides/gha.md) – Learn how GitHub Actions automate your workflows -- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) – Set up container builds with GitHub Actions -- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) – Full reference for writing GitHub workflows -- [Compose file reference](/compose/compose-file/) – Full configuration reference for `compose.yaml` -- [Best practices for writing Dockerfiles](/develop/develop-images/dockerfile_best-practices/) – Optimize your image for performance and security - ---- - -## Next steps - -Next, learn how you can locally test and debug your Next.js workloads on Kubernetes before deploying. This helps you ensure your application behaves as expected in a production-like environment, reducing surprises during deployment. diff --git a/content/guides/nextjs/containerize.md b/content/guides/nextjs/containerize.md deleted file mode 100644 index 2fac91c48af3..000000000000 --- a/content/guides/nextjs/containerize.md +++ /dev/null @@ -1,961 +0,0 @@ ---- -title: Containerize a Next.js Application -linkTitle: Containerize -weight: 10 -keywords: next.js, node, image, initialize, build -description: Learn how to containerize a Next.js application with Docker by creating an optimized, production-ready image using best practices for performance, security, and scalability. - ---- - -## Prerequisites - -Before you begin, make sure the following tools are installed and available on your system: - -- You have installed the latest version of [Docker Desktop](/get-started/get-docker.md). -- You have a [git client](https://git-scm.com/downloads). The examples in this section use a command-line based git client, but you can use any client. - -> [!NOTE] -> New to Docker? Start with the [Docker basics](/get-started/docker-concepts/the-basics/what-is-a-container.md) guide to get familiar with key concepts like images, containers, and Dockerfiles. - ---- - -## Overview - -This guide walks you through containerizing a Next.js application with Docker. -You'll learn how to create a production-ready Docker image using best -practices that improve performance, security, scalability, and deployment -efficiency. - -By the end of this guide, you will: - -- Containerize a Next.js application using Docker. -- Create and optimize a Dockerfile for production builds. -- Use multi-stage builds to minimize image size. -- Leverage Next.js standalone or export output for efficient containerization. -- Follow best practices for building secure and maintainable Docker images. - ---- - -## Get the sample application - -Clone the sample application to use with this guide. Open a terminal, change -directory to a directory that you want to work in, and run the following command -to clone the git repository: - -```console -$ git clone https://github.com/kristiyan-velkov/docker-nextjs-sample -``` ---- - -## Build the Docker image - -Next.js has specific requirements for production deployments. This guide shows two approaches: `standalone` output (Node.js server) and `export` output (static files with Nginx). - -> [!TIP] -> -> [Gordon](/ai/gordon/), Docker's AI assistant, can generate Docker assets for your project. Ask Gordon to create a Dockerfile, Compose file, and `.dockerignore` tailored to your application. - -### Step 1: Configure Next.js and create the Dockerfile - -Before creating a Dockerfile, choose a base image: the [Node.js Official Image](https://hub.docker.com/_/node) or a [Docker Hardened Image (DHI)](https://hub.docker.com/hardened-images/catalog) from the Hardened Image catalog. Choosing DHI gives you a production-ready, lightweight, and secure image. For more information, see [Docker Hardened Images](https://docs.docker.com/dhi/). - -> [!IMPORTANT] -> This guide uses stable Node.js LTS image tags that are considered secure when the guide is written. Because new releases and security patches are published regularly, always review the [official Node.js Docker images](https://hub.docker.com/_/node) and select a secure, up-to-date version before building or deploying. - ---- - -#### 1.1 Next.js with standalone output - -Standalone output (`output: "standalone"`) makes Next.js build a self-contained output that includes only the files and dependencies needed to run the application. A single `node server.js` can serve the app, which is ideal for Docker and supports server-side rendering, API routes, and incremental static regeneration. For details, see the [Next.js output configuration documentation](https://nextjs.org/docs/app/api-reference/config/next-config-js/output) (including the "standalone" option). - -The container runs the Next.js server with Node.js on port 3000. - -Configure Next.js — Open or create `next.config.ts` in your project root: - -```ts -import type { NextConfig } from "next"; - -const nextConfig: NextConfig = { - output: "standalone", -}; - -export default nextConfig; -``` - -Choose either a Docker Hardened Image or the Docker Official Image, then create a `Dockerfile` using the content from the selected tab below. - -{{< tabs >}} -{{< tab name="Using Docker Hardened Images" >}} - -Docker Hardened Images (DHIs) are available for Node.js in the [Docker Hardened Images catalog](https://hub.docker.com/hardened-images/catalog/dhi/node). For more information, see the [DHI quickstart](/dhi/get-started/) guide. - -1. Sign in to the DHI registry: - ```console - $ docker login dhi.io - ``` - -2. Pull the Node.js DHI (check the catalog for available versions): - ```console - $ docker pull dhi.io/node:24-alpine3.22-dev - ``` - -3. Create a file named `Dockerfile` with the following contents. The `FROM` instructions use `dhi.io/node:24-alpine3.22-dev`. Check the [Docker Hardened Images catalog](https://hub.docker.com/hardened-images/catalog) for the latest versions and update the image tags as needed for security and compatibility. - - ```dockerfile - # ============================================ - # Stage 1: Dependencies Installation Stage - # ============================================ - - # IMPORTANT: Docker Hardened Image (DHI) Version Maintenance - # This Dockerfile uses dhi.io/node. Regularly validate and update to the latest DHI versions in the catalog for security and compatibility. - - FROM dhi.io/node:24-alpine3.22-dev AS dependencies - - # Set working directory - WORKDIR /app - - # Copy package-related files first to leverage Docker's caching mechanism - COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml* .npmrc* ./ - - # Install project dependencies with frozen lockfile for reproducible builds - RUN --mount=type=cache,target=/root/.npm \ - --mount=type=cache,target=/usr/local/share/.cache/yarn \ - --mount=type=cache,target=/root/.local/share/pnpm/store \ - if [ -f package-lock.json ]; then \ - npm ci --no-audit --no-fund; \ - elif [ -f yarn.lock ]; then \ - corepack enable yarn && yarn install --frozen-lockfile --production=false; \ - elif [ -f pnpm-lock.yaml ]; then \ - corepack enable pnpm && pnpm install --frozen-lockfile; \ - else \ - echo "No lockfile found." && exit 1; \ - fi - - # ============================================ - # Stage 2: Build Next.js application in standalone mode - # ============================================ - - FROM dhi.io/node:24-alpine3.22-dev AS builder - - # Set working directory - WORKDIR /app - - # Copy project dependencies from dependencies stage - COPY --from=dependencies /app/node_modules ./node_modules - - # Copy application source code - COPY . . - - ENV NODE_ENV=production - - # Next.js collects completely anonymous telemetry data about general usage. - # Learn more here: https://nextjs.org/telemetry - # Uncomment the following line in case you want to disable telemetry during the build. - # ENV NEXT_TELEMETRY_DISABLED=1 - - # Build Next.js application - # If you want to speed up Docker rebuilds, you can cache the build artifacts - # by adding: --mount=type=cache,target=/app/.next/cache - # This caches the .next/cache directory across builds, but it also prevents - # .next/cache/fetch-cache from being included in the final image, meaning - # cached fetch responses from the build won't be available at runtime. - RUN if [ -f package-lock.json ]; then \ - npm run build; \ - elif [ -f yarn.lock ]; then \ - corepack enable yarn && yarn build; \ - elif [ -f pnpm-lock.yaml ]; then \ - corepack enable pnpm && pnpm build; \ - else \ - echo "No lockfile found." && exit 1; \ - fi - - # ============================================ - # Stage 3: Run Next.js application - # ============================================ - - FROM dhi.io/node:24-alpine3.22-dev AS runner - - # Set working directory - WORKDIR /app - - # Set production environment variables - ENV NODE_ENV=production - ENV PORT=3000 - ENV HOSTNAME="0.0.0.0" - - # Next.js collects completely anonymous telemetry data about general usage. - # Learn more here: https://nextjs.org/telemetry - # Uncomment the following line in case you want to disable telemetry during the run time. - # ENV NEXT_TELEMETRY_DISABLED=1 - - # Copy production assets - COPY --from=builder --chown=node:node /app/public ./public - - # Set the correct permission for prerender cache - RUN mkdir .next - RUN chown node:node .next - - # Automatically leverage output traces to reduce image size - # https://nextjs.org/docs/advanced-features/output-file-tracing - COPY --from=builder --chown=node:node /app/.next/standalone ./ - COPY --from=builder --chown=node:node /app/.next/static ./.next/static - - # If you want to persist the fetch cache generated during the build so that - # cached responses are available immediately on startup, uncomment this line: - # COPY --from=builder --chown=node:node /app/.next/cache ./.next/cache - - # Switch to non-root user for security best practices - USER node - - # Expose port 3000 to allow HTTP traffic - EXPOSE 3000 - - # Start Next.js standalone server - CMD ["node", "server.js"] - ``` - -{{< /tab >}} -{{< tab name="Using the Docker Official Image" >}} - -Create a file named `Dockerfile` with the following contents (uses `node`): - -```dockerfile - # ============================================ - # Stage 1: Dependencies Installation Stage - # ============================================ - - ARG NODE_VERSION=24.14.0-slim - - FROM node:${NODE_VERSION} AS dependencies - - # Set working directory - WORKDIR /app - - # Copy package-related files first to leverage Docker's caching mechanism - COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml* .npmrc* ./ - - # Install project dependencies with frozen lockfile for reproducible builds - RUN --mount=type=cache,target=/root/.npm \ - --mount=type=cache,target=/usr/local/share/.cache/yarn \ - --mount=type=cache,target=/root/.local/share/pnpm/store \ - if [ -f package-lock.json ]; then \ - npm ci --no-audit --no-fund; \ - elif [ -f yarn.lock ]; then \ - corepack enable yarn && yarn install --frozen-lockfile --production=false; \ - elif [ -f pnpm-lock.yaml ]; then \ - corepack enable pnpm && pnpm install --frozen-lockfile; \ - else \ - echo "No lockfile found." && exit 1; \ - fi - - # ============================================ - # Stage 2: Build Next.js application in standalone mode - # ============================================ - - FROM node:${NODE_VERSION} AS builder - - # Set working directory - WORKDIR /app - - # Copy project dependencies from dependencies stage - COPY --from=dependencies /app/node_modules ./node_modules - - # Copy application source code - COPY . . - - ENV NODE_ENV=production - - # Next.js collects completely anonymous telemetry data about general usage. - # Learn more here: https://nextjs.org/telemetry - # Uncomment the following line in case you want to disable telemetry during the build. - # ENV NEXT_TELEMETRY_DISABLED=1 - - # Build Next.js application - # If you want to speed up Docker rebuilds, you can cache the build artifacts - # by adding: --mount=type=cache,target=/app/.next/cache - # This caches the .next/cache directory across builds, but it also prevents - # .next/cache/fetch-cache from being included in the final image, meaning - # cached fetch responses from the build won't be available at runtime. - RUN if [ -f package-lock.json ]; then \ - npm run build; \ - elif [ -f yarn.lock ]; then \ - corepack enable yarn && yarn build; \ - elif [ -f pnpm-lock.yaml ]; then \ - corepack enable pnpm && pnpm build; \ - else \ - echo "No lockfile found." && exit 1; \ - fi - - # ============================================ - # Stage 3: Run Next.js application - # ============================================ - - FROM node:${NODE_VERSION} AS runner - - # Set working directory - WORKDIR /app - - # Set production environment variables - ENV NODE_ENV=production - ENV PORT=3000 - ENV HOSTNAME="0.0.0.0" - - # Next.js collects completely anonymous telemetry data about general usage. - # Learn more here: https://nextjs.org/telemetry - # Uncomment the following line in case you want to disable telemetry during the run time. - # ENV NEXT_TELEMETRY_DISABLED=1 - - # Copy production assets - COPY --from=builder --chown=node:node /app/public ./public - - # Set the correct permission for prerender cache - RUN mkdir .next - RUN chown node:node .next - - # Automatically leverage output traces to reduce image size - # https://nextjs.org/docs/advanced-features/output-file-tracing - COPY --from=builder --chown=node:node /app/.next/standalone ./ - COPY --from=builder --chown=node:node /app/.next/static ./.next/static - - # If you want to persist the fetch cache generated during the build so that - # cached responses are available immediately on startup, uncomment this line: - # COPY --from=builder --chown=node:node /app/.next/cache ./.next/cache - - # Switch to non-root user for security best practices - USER node - - # Expose port 3000 to allow HTTP traffic - EXPOSE 3000 - - # Start Next.js standalone server - CMD ["node", "server.js"] -``` - -> [!NOTE] -> This Dockerfile uses three stages: `dependencies`, `builder`, and `runner`. The final image runs `node server.js` and listens on port 3000. - -{{< /tab >}} -{{< /tabs >}} - ---- - -#### 1.2 Next.js with export output - -Output export (`output: "export"`) makes Next.js build a fully static site at build time. It generates HTML, CSS, and JavaScript into an `out` directory that can be served by any static host or CDN—no Node.js server at runtime. Use this when you don't need server-side rendering or API routes. For details, see the [Next.js output configuration documentation](https://nextjs.org/docs/app/api-reference/config/next-config-js/output). - -Configure Next.js — Open `next.config.ts` in your project root and add the following code: - -```ts -import type { NextConfig } from "next"; - -const nextConfig: NextConfig = { - output: "export", - trailingSlash: true, - images: { - unoptimized: true, - }, -}; - -export default nextConfig; -``` - -Choose either a Docker Hardened Image or the Docker Official Image, then create a `Dockerfile` using the content from the selected tab below. - -{{< tabs >}} -{{< tab name="Using Docker Hardened Images" >}} - -Docker Hardened Images (DHIs) are available for Node.js and Nginx in the [Docker Hardened Images catalog](https://hub.docker.com/hardened-images/catalog). For more information, see the [DHI quickstart](/dhi/get-started/) guide. - -1. Sign in to the DHI registry: - ```console - $ docker login dhi.io - ``` - -2. Pull the Node.js DHI (check the catalog for available versions): - ```console - $ docker pull dhi.io/node:24-alpine3.22-dev - ``` - -3. Pull the Nginx DHI (check the catalog for available versions): - ```console - $ docker pull dhi.io/nginx:1.28.0-alpine3.21-dev - ``` - -4. Create a file named `Dockerfile` with the following contents. The `FROM` instructions use Docker Hardened Images: `dhi.io/node:24-alpine3.22-dev` and `dhi.io/nginx:1.28.0-alpine3.21-dev`. Check the [Docker Hardened Images catalog](https://hub.docker.com/hardened-images/catalog) for the latest versions and update the image tags as needed for security and compatibility. - - ```dockerfile - # ============================================ - # Stage 1: Dependencies Installation Stage - # ============================================ - - # IMPORTANT: Docker Hardened Image (DHI) Version Maintenance - # This Dockerfile uses dhi.io/node and dhi.io/nginx. Regularly validate and update to the latest DHI versions in the catalog for security and compatibility. - - FROM dhi.io/node:24-alpine3.22-dev AS dependencies - - # Set the working directory - WORKDIR /app - - # Copy package-related files first to leverage Docker's caching mechanism - COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml* .npmrc* ./ - - # Install project dependencies with frozen lockfile for reproducible builds - RUN --mount=type=cache,target=/root/.npm \ - --mount=type=cache,target=/usr/local/share/.cache/yarn \ - --mount=type=cache,target=/root/.local/share/pnpm/store \ - if [ -f package-lock.json ]; then \ - npm ci --no-audit --no-fund; \ - elif [ -f yarn.lock ]; then \ - corepack enable yarn && yarn install --frozen-lockfile --production=false; \ - elif [ -f pnpm-lock.yaml ]; then \ - corepack enable pnpm && pnpm install --frozen-lockfile; \ - else \ - echo "No lockfile found." && exit 1; \ - fi - - # ============================================ - # Stage 2: Build Next.js Application - # ============================================ - - FROM dhi.io/node:24-alpine3.22-dev AS builder - - # Set the working directory - WORKDIR /app - - # Copy project dependencies from dependencies stage - COPY --from=dependencies /app/node_modules ./node_modules - - # Copy application source code - COPY . . - - ENV NODE_ENV=production - - # Next.js collects completely anonymous telemetry data about general usage. - # Learn more here: https://nextjs.org/telemetry - # Uncomment the following line in case you want to disable telemetry during the build. - # ENV NEXT_TELEMETRY_DISABLED=1 - - # Build Next.js application - RUN --mount=type=cache,target=/app/.next/cache \ - if [ -f package-lock.json ]; then \ - npm run build; \ - elif [ -f yarn.lock ]; then \ - corepack enable yarn && yarn build; \ - elif [ -f pnpm-lock.yaml ]; then \ - corepack enable pnpm && pnpm build; \ - else \ - echo "No lockfile found." && exit 1; \ - fi - - # ========================================= - # Stage 3: Serve Static Files with Nginx - # ========================================= - - FROM dhi.io/nginx:1.28.0-alpine3.21-dev AS runner - - # Set the working directory - WORKDIR /app - - # Next.js collects completely anonymous telemetry data about general usage. - # Learn more here: https://nextjs.org/telemetry - # Uncomment the following line in case you want to disable telemetry during the run time. - # ENV NEXT_TELEMETRY_DISABLED=1 - - # Copy custom Nginx config - COPY nginx.conf /etc/nginx/nginx.conf - - # Copy the static build output from the build stage to Nginx's default HTML serving directory - COPY --chown=nginx:nginx --from=builder /app/out /usr/share/nginx/html - - # Non-root user for security best practices - USER nginx - - # Expose port 8080 to allow HTTP traffic - EXPOSE 8080 - - # Start Nginx directly with custom config - ENTRYPOINT ["nginx", "-c", "/etc/nginx/nginx.conf"] - CMD ["-g", "daemon off;"] - ``` - -{{< /tab >}} -{{< tab name="Using the Docker Official Image" >}} - -Create a file named `Dockerfile` with the following contents (uses `node` and `nginxinc/nginx-unprivileged`): - -```dockerfile -# ============================================ -# Stage 1: Dependencies Installation Stage -# ============================================ - -ARG NODE_VERSION=24.14.0-slim -ARG NGINXINC_IMAGE_TAG=alpine3.22 - -FROM node:${NODE_VERSION} AS dependencies - -# Set the working directory -WORKDIR /app - -# Copy package-related files first to leverage Docker's caching mechanism -COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml* .npmrc* ./ - -# Install project dependencies with frozen lockfile for reproducible builds -RUN --mount=type=cache,target=/root/.npm \ - --mount=type=cache,target=/usr/local/share/.cache/yarn \ - --mount=type=cache,target=/root/.local/share/pnpm/store \ - if [ -f package-lock.json ]; then \ - npm ci --no-audit --no-fund; \ - elif [ -f yarn.lock ]; then \ - corepack enable yarn && yarn install --frozen-lockfile --production=false; \ - elif [ -f pnpm-lock.yaml ]; then \ - corepack enable pnpm && pnpm install --frozen-lockfile; \ - else \ - echo "No lockfile found." && exit 1; \ - fi - -# ============================================ -# Stage 2: Build Next.js Application -# ============================================ - -FROM node:${NODE_VERSION} AS builder - -# Set the working directory -WORKDIR /app - -# Copy project dependencies from dependencies stage -COPY --from=dependencies /app/node_modules ./node_modules - -# Copy application source code -COPY . . - -ENV NODE_ENV=production - -# Next.js collects completely anonymous telemetry data about general usage. -# Learn more here: https://nextjs.org/telemetry -# Uncomment the following line in case you want to disable telemetry during the build. -# ENV NEXT_TELEMETRY_DISABLED=1 - -# Build Next.js application -RUN --mount=type=cache,target=/app/.next/cache \ - if [ -f package-lock.json ]; then \ - npm run build; \ - elif [ -f yarn.lock ]; then \ - corepack enable yarn && yarn build; \ - elif [ -f pnpm-lock.yaml ]; then \ - corepack enable pnpm && pnpm build; \ - else \ - echo "No lockfile found." && exit 1; \ - fi - -# ========================================= -# Stage 3: Serve Static Files with Nginx -# ========================================= - -FROM nginxinc/nginx-unprivileged:${NGINXINC_IMAGE_TAG} AS runner - -# Set the working directory -WORKDIR /app - -# Next.js collects completely anonymous telemetry data about general usage. -# Learn more here: https://nextjs.org/telemetry -# Uncomment the following line in case you want to disable telemetry during the run time. -# ENV NEXT_TELEMETRY_DISABLED=1 - -# Copy custom Nginx config -COPY nginx.conf /etc/nginx/nginx.conf - -# Copy the static build output from the build stage to Nginx's default HTML serving directory -COPY --from=builder /app/out /usr/share/nginx/html - -# Non-root user for security best practices -USER nginx - -# Expose port 8080 to allow HTTP traffic -EXPOSE 8080 - -# Start Nginx directly with custom config -ENTRYPOINT ["nginx", "-c", "/etc/nginx/nginx.conf"] -CMD ["-g", "daemon off;"] -``` - -> [!NOTE] -> This guide uses [nginx-unprivileged](https://hub.docker.com/r/nginxinc/nginx-unprivileged) instead of the standard Nginx image to run as a non-root user, following security best practices. - -{{< /tab >}} -{{< /tabs >}} - -1. Create `nginx.conf` (required for export output only) — Create a file named `nginx.conf` in the root of your project: - - ```nginx - # Minimal Nginx config for static Next.js app - worker_processes 1; - - # Store PID in /tmp (always writable) - pid /tmp/nginx.pid; - - events { - worker_connections 1024; - } - - http { - include /etc/nginx/mime.types; - default_type application/octet-stream; - - # Disable logging to avoid permission issues - access_log off; - error_log /dev/stderr; - - # Optimize static file serving - sendfile on; - tcp_nopush on; - tcp_nodelay on; - keepalive_timeout 65; - - # Gzip compression - gzip on; - gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript; - gzip_min_length 256; - - server { - listen 8080; - server_name localhost; - - # Serve static files - root /usr/share/nginx/html; - index index.html; - - # Handle Next.js static export routing - # See: https://nextjs.org/docs/app/guides/static-exports#deploying - location / { - try_files $uri $uri.html $uri/ =404; - } - - # This is necessary when `trailingSlash: false` (default). - # You can omit this when `trailingSlash: true` in next.config. - # Handles nested routes like /blog/post -> /blog/post.html - location ~ ^/(.+)/$ { - rewrite ^/(.+)/$ /$1.html break; - } - - # Serve Next.js static assets - location ~ ^/_next/ { - try_files $uri =404; - expires 1y; - add_header Cache-Control "public, immutable"; - } - - # Optional 404 handling - error_page 404 /404.html; - location = /404.html { - internal; - } - } - } - ``` - - > [!NOTE] - > Export uses port 8080. For more details, see the [Next.js output configuration](https://nextjs.org/docs/app/api-reference/config/next-config-js/output) and [Nginx documentation](https://nginx.org/en/docs/). - -### Step 2: Create the compose.yaml file - -Create a file named `compose.yaml` with the following contents: - -```yaml {collapse=true,title=compose.yaml} -services: - server: - build: - context: . - ports: - - 3000:3000 -``` - -> [!NOTE] -> If using export output (Nginx), change the port mapping to `8080:8080`. - -### Step 3: Create the .dockerignore file - -The `.dockerignore` file tells Docker which files and folders to exclude when building the image. - - -> [!NOTE] ->This helps: ->- Reduce image size ->- Speed up the build process ->- Prevent sensitive or unnecessary files (like `.env`, `.git`, or `node_modules`) from being added to the final image. -> -> To learn more, visit the [.dockerignore reference](/reference/dockerfile.md#dockerignore-file). - -Create a file named `.dockerignore` with the following contents: - -```dockerignore -# Dependencies (installed inside the image, never copy from host) -node_modules/ -.pnp/ -.pnp.js -.pnpm-store/ - -# Next.js build output (generated during the image build) -.next/ -out/ -dist/ -build/ -.vercel/ - -# Testing (not needed in the production image) -coverage/ -.nyc_output/ -__tests__/ -__mocks__/ -jest/ -cypress/ -playwright-report/ -test-results/ -.vitest/ - -# Environment files (avoid leaking secrets into the build context) -.env -.env* -.env.local -.env.development.local -.env.test.local -.env.production.local - -# Debug and log files -npm-debug.log* -yarn-debug.log* -yarn-error.log* -pnpm-debug.log* -lerna-debug.log* -*.log - -# IDE and editor files -.vscode/ -.idea/ -.cursor/ -.cursorrules -.copilot/ -*.swp -*.swo -*~ - -# Git -.git/ -.gitignore -.gitattributes - -# Docker files (reduce build context; not needed inside the image) -Dockerfile* -.dockerignore -docker-compose*.yml -compose*.yaml - -# Documentation (not needed in the image) -*.md -docs/ - -# CI/CD (not needed in the image) -.github/ -.gitlab-ci.yml -.travis.yml -.circleci/ -Jenkinsfile - -# TypeScript and build metadata -*.tsbuildinfo - -# Cache and temporary directories -.cache/ -.parcel-cache/ -.eslintcache -.stylelintcache -.turbo/ -.tmp/ -.temp/ - -# Sensitive or dev-only config (optional; omit if your build needs these) -.pem -.editorconfig -.prettierrc* -.eslintrc* -.stylelintrc* -.babelrc* -*.iml - -# OS-specific files -.DS_Store -._* -.Spotlight-V100 -.Trashes -ehthumbs.db -Thumbs.db -Desktop.ini -``` - -### Step 4: Build the Next.js application image - -With your custom configuration in place, you're now ready to build the Docker image. Use the Dockerfile you created in Step 1 (standalone or export). - -The setup includes: - -- Multi-stage builds for optimized image size -- Standalone: Node.js server on port 3000; Export: Nginx serving static files on port 8080 -- Non-root user for enhanced security -- Proper file permissions and ownership - -After completing the previous steps, your project directory should contain at least the following files (export also requires `nginx.conf`): - -```text -├── docker-nextjs-sample/ -│ ├── Dockerfile -│ ├── .dockerignore -│ ├── compose.yaml -│ └── next.config.ts -``` - -Now that your Dockerfile is configured, you can build the Docker image for your Next.js application. - -> [!NOTE] -> The `docker build` command packages your application into an image using the instructions in the Dockerfile. It includes all necessary files from the current directory (called the [build context](/build/concepts/context/#what-is-a-build-context)). - -Run the following command from the root of your project: - -```console -$ docker build --tag nextjs-sample . -``` - -What this command does: -- Uses the Dockerfile in the current directory (.) -- Packages the application and its dependencies into a Docker image -- Tags the image as nextjs-sample so you can reference it later - - -### Step 5: View local images - -After building your Docker image, you can check which images are available on your local machine using either the Docker CLI or [Docker Desktop](/manuals/desktop/use-desktop/images.md). Since you're already working in the terminal, let's use the Docker CLI. - -To list all locally available Docker images, run the following command: - -```console -$ docker images -``` - -Example Output: - -```shell -REPOSITORY TAG IMAGE ID CREATED SIZE -nextjs-sample latest 8c5fc80f098e 14 seconds ago 130MB -``` - -This output provides key details about your images: - -- Repository – The name assigned to the image. -- Tag – A version label that helps identify different builds (e.g., latest). -- Image ID – A unique identifier for the image. -- Created – The timestamp indicating when the image was built. -- Size – The total disk space used by the image. - -If the build was successful, you should see `nextjs-sample` image listed. - ---- - -## Run the containerized application - -In the previous step, you created a Dockerfile for your Next.js application and built a Docker image using the docker build command. Now it's time to run that image in a container and verify that your application works as expected. - -Run the following command in a terminal. Use the port that matches your setup: standalone uses port 3000, export uses port 8080. - -```console -$ docker run -p 3000:3000 nextjs-sample -``` - -For export output, use port 8080 instead: - -```console -$ docker run -p 8080:8080 nextjs-sample -``` - -Open a browser and view the application: [http://localhost:3000](http://localhost:3000) for standalone or [http://localhost:8080](http://localhost:8080) for export. You should see your Next.js web application. - -Press `ctrl+c` in the terminal to stop your application. - -### Run the application in the background - -You can run the application detached from the terminal by adding the `-d` option and `--name` to give the container a name so you can stop it later: - -```console -$ docker run -d -p 3000:3000 --name nextjs-app nextjs-sample -``` - -For export output, use port 8080: - -```console -$ docker run -d -p 8080:8080 --name nextjs-app nextjs-sample -``` - -Open a browser and view the application: [http://localhost:3000](http://localhost:3000) for standalone or [http://localhost:8080](http://localhost:8080) for export. You should see your web application. - -To confirm that the container is running, use the `docker ps` command: - -```console -$ docker ps -``` - -This will list all active containers along with their ports, names, and status. Look for a container exposing port 3000 (standalone) or 8080 (export). - -Example Output: - -```shell -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -f49b74736a9d nextjs-sample "node server.js" About a minute ago Up About a minute 0.0.0.0:3000->3000/tcp nextjs-app -``` - -To stop the application, run: - -```console -$ docker stop nextjs-app -``` - -> [!NOTE] -> For more information about running containers, see the [`docker run` CLI reference](/reference/cli/docker/container/run/) and the [`docker stop` CLI reference](/reference/cli/docker/container/stop/). - ---- - -## Summary - -In this guide, you learned how to containerize, build, and run a Next.js application using Docker. By following best practices, you created a secure, optimized, and production-ready setup. - -What you accomplished: -- Configured Next.js for either standalone output (Node.js server) or export output (static files with Nginx). -- Added a multi-stage Dockerfile for your chosen approach: standalone (port 3000) or export (port 8080, with `nginx.conf`). -- Created a `.dockerignore` file to exclude unnecessary files and keep the image clean and efficient. -- Built your Docker image using `docker build`. -- Ran the container using `docker run` with the image name `nextjs-sample`, both in the foreground and in detached mode. -- Verified that the app was running by visiting [http://localhost:3000](http://localhost:3000) (standalone) or [http://localhost:8080](http://localhost:8080) (export). -- Learned how to stop the containerized application using `docker stop nextjs-app`. - -You now have a fully containerized Next.js application, running in a Docker container, and ready for deployment across any environment with confidence and consistency. - ---- - -## Related resources - -Explore official references and best practices to sharpen your Docker workflow: - -- [Multi-stage builds](/build/building/multi-stage/) – Learn how to separate build and runtime stages. -- [Best practices for writing Dockerfiles](/develop/develop-images/dockerfile_best-practices/) – Write efficient, maintainable, and secure Dockerfiles. -- [Build context in Docker](/build/concepts/context/) – Learn how context affects image builds. -- [Next.js output configuration](https://nextjs.org/docs/app/api-reference/config/next-config-js/output) – Learn about Next.js production optimization (standalone and export). -- [Next.js with Docker (standalone)](https://github.com/vercel/next.js/tree/canary/examples/with-docker) – Official Next.js example: standalone output with Node.js. -- [Next.js with Docker (export)](https://github.com/vercel/next.js/tree/canary/examples/with-docker-export-output) – Official Next.js example: static export with Nginx or serve. -- [`docker build` CLI reference](/reference/cli/docker/image/build/) – Build Docker images from a Dockerfile. -- [`docker images` CLI reference](/reference/cli/docker/image/ls/) – Manage and inspect local Docker images. -- [`docker run` CLI reference](/reference/cli/docker/container/run/) – Run a command in a new container. -- [`docker stop` CLI reference](/reference/cli/docker/container/stop/) – Stop one or more running containers. - ---- - -## Next steps - -With your Next.js application now containerized, you're ready to move on to the next step. - -In the next section, you'll learn how to develop your application using Docker containers, enabling a consistent, isolated, and reproducible development environment across any machine. diff --git a/content/guides/nextjs/deploy.md b/content/guides/nextjs/deploy.md deleted file mode 100644 index 7b1017e2c8bb..000000000000 --- a/content/guides/nextjs/deploy.md +++ /dev/null @@ -1,199 +0,0 @@ ---- -title: Test your Next.js deployment -linkTitle: Test your deployment -weight: 70 -keywords: deploy, kubernetes, next.js -description: Learn how to deploy locally to test and debug your Kubernetes deployment - ---- - -## Prerequisites - -Before you begin, make sure you've completed the following: -- Complete all the previous sections of this guide, starting with [Containerize Next.js application](containerize.md). -- [Enable Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. - -> **New to Kubernetes?** -> Visit the [Kubernetes basics tutorial](https://kubernetes.io/docs/tutorials/kubernetes-basics/) to get familiar with how clusters, pods, deployments, and services work. - ---- - -## Overview - -This section guides you through deploying your containerized Next.js application locally using [Docker Desktop's built-in Kubernetes](/desktop/kubernetes/). Running your app in a local Kubernetes cluster allows you to closely simulate a real production environment, enabling you to test, validate, and debug your workloads with confidence before promoting them to staging or production. - ---- - -## Create a Kubernetes YAML file - -Follow these steps to define your deployment configuration: - -1. In the root of your project, create a new file named: nextjs-sample-kubernetes.yaml - -2. Open the file in your IDE or preferred text editor. - -3. Add the following configuration, and be sure to replace `{DOCKER_USERNAME}` and `{DOCKERHUB_PROJECT_NAME}` with your actual Docker Hub username and repository name from the previous [Automate your builds with GitHub Actions](configure-github-actions.md). - - -```yaml -apiVersion: apps/v1 -kind: Deployment -metadata: - name: nextjs-sample - namespace: default -spec: - replicas: 1 - selector: - matchLabels: - app: nextjs-sample - template: - metadata: - labels: - app: nextjs-sample - spec: - containers: - - name: nextjs-container - image: {DOCKER_USERNAME}/{DOCKERHUB_PROJECT_NAME}:latest - imagePullPolicy: Always - ports: - - containerPort: 3000 - env: - - name: NODE_ENV - value: "production" - - name: HOSTNAME - value: "0.0.0.0" ---- -apiVersion: v1 -kind: Service -metadata: - name: nextjs-sample-service - namespace: default -spec: - type: NodePort - selector: - app: nextjs-sample - ports: - - port: 3000 - targetPort: 3000 - nodePort: 30001 -``` - -This manifest defines two key Kubernetes resources, separated by `---`: - -- Deployment - Deploys a single replica of your Next.js application inside a pod. The pod uses the Docker image built and pushed by your GitHub Actions CI/CD workflow - (refer to [Automate your builds with GitHub Actions](configure-github-actions.md)). - The container listens on port `3000`, which is the default port for Next.js applications. - -- Service (NodePort) - Exposes the deployed pod to your local machine. - It forwards traffic from port `30001` on your host to port `3000` inside the container. - This lets you access the application in your browser at [http://localhost:30001](http://localhost:30001). - -> [!NOTE] -> To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). - ---- - -## Deploy and check your application - -Follow these steps to deploy your containerized Next.js app into a local Kubernetes cluster and verify that it's running correctly. - -### Step 1. Apply the Kubernetes configuration - -In your terminal, navigate to the directory where your `nextjs-sample-kubernetes.yaml` file is located, then deploy the resources using: - -```console - $ kubectl apply -f nextjs-sample-kubernetes.yaml -``` - -If everything is configured properly, you'll see confirmation that both the Deployment and the Service were created: - -```shell - deployment.apps/nextjs-sample created - service/nextjs-sample-service created -``` - -This output means that both the Deployment and the Service were successfully created and are now running inside your local cluster. - -### Step 2. Check the deployment status - -Run the following command to check the status of your deployment: - -```console - $ kubectl get deployments -``` - -You should see an output similar to: - -```shell - NAME READY UP-TO-DATE AVAILABLE AGE - nextjs-sample 1/1 1 1 14s -``` - -This confirms that your pod is up and running with one replica available. - -### Step 3. Verify the service exposure - -Check if the NodePort service is exposing your app to your local machine: - -```console -$ kubectl get services -``` - -You should see something like: - -```shell -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -nextjs-sample-service NodePort 10.100.244.65 3000:30001/TCP 1m -``` - -This output confirms that your app is available via NodePort on port 30001. - -### Step 4. Access your app in the browser - -Open your browser and navigate to [http://localhost:30001](http://localhost:30001). - -You should see your production-ready Next.js Sample application running — served by your local Kubernetes cluster. - -### Step 5. Clean up Kubernetes resources - -Once you're done testing, you can delete the deployment and service using: - -```console - $ kubectl delete -f nextjs-sample-kubernetes.yaml -``` - -Expected output: - -```shell - deployment.apps "nextjs-sample" deleted - service "nextjs-sample-service" deleted -``` - -This ensures your cluster stays clean and ready for the next deployment. - ---- - -## Summary - -In this section, you learned how to deploy your Next.js application to a local Kubernetes cluster using Docker Desktop. This setup allows you to test and debug your containerized app in a production-like environment before deploying it to the cloud. - -What you accomplished: - -- Created a Kubernetes Deployment and NodePort Service for your Next.js app -- Used `kubectl apply` to deploy the application locally -- Verified the app was running and accessible at `http://localhost:30001` -- Cleaned up your Kubernetes resources after testing - ---- - -## Related resources - -Explore official references and best practices to sharpen your Kubernetes deployment workflow: - -- [Kubernetes documentation](https://kubernetes.io/docs/home/) – Learn about core concepts, workloads, services, and more. -- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) – Use Docker Desktop's built-in Kubernetes support for local testing and development. -- [`kubectl` CLI reference](https://kubernetes.io/docs/reference/kubectl/) – Manage Kubernetes clusters from the command line. -- [Kubernetes Deployment resource](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) – Understand how to manage and scale applications using Deployments. -- [Kubernetes Service resource](https://kubernetes.io/docs/concepts/services-networking/service/) – Learn how to expose your application to internal and external traffic. diff --git a/content/guides/nextjs/develop.md b/content/guides/nextjs/develop.md deleted file mode 100644 index d2a758434e97..000000000000 --- a/content/guides/nextjs/develop.md +++ /dev/null @@ -1,209 +0,0 @@ ---- -title: Use containers for Next.js development -linkTitle: Develop your app -weight: 30 -keywords: next.js, development, node -description: Learn how to develop your Next.js application locally using containers. - ---- - -## Prerequisites - -Complete [Containerize Next.js application](containerize.md). - ---- - -## Overview - -In this section, you'll learn how to set up both production and development environments for your containerized Next.js application using Docker Compose. This setup allows you to run a production build using the standalone server and to develop efficiently inside containers using Next.js's built-in hot reloading with Compose Watch. - -You'll learn how to: -- Configure separate containers for production and development -- Enable automatic file syncing using Compose Watch in development -- Debug and live-preview your changes in real-time without manual rebuilds - ---- - -## Automatically update services (development mode) - -Use Compose Watch to automatically sync source file changes into your -containerized development environment. This automatically syncs file changes -without needing to restart or rebuild containers manually. - -## Step 1: Create a development Dockerfile - -Create a file named `Dockerfile.dev` in your project root with the following content (matching the [sample project](https://github.com/kristiyan-velkov/docker-nextjs-sample)): - -```dockerfile -# ============================================ -# Development Dockerfile for Next.js -# ============================================ -ARG NODE_VERSION=24.14.0-slim - -FROM node:${NODE_VERSION} AS dev - -WORKDIR /app - -COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml* .npmrc* ./ - -RUN --mount=type=cache,target=/root/.npm \ - --mount=type=cache,target=/usr/local/share/.cache/yarn \ - --mount=type=cache,target=/root/.local/share/pnpm/store \ - if [ -f package-lock.json ]; then \ - npm ci --no-audit --no-fund; \ - elif [ -f yarn.lock ]; then \ - corepack enable yarn && yarn install --frozen-lockfile --production=false; \ - elif [ -f pnpm-lock.yaml ]; then \ - corepack enable pnpm && pnpm install --frozen-lockfile; \ - else \ - echo "No lockfile found." && exit 1; \ - fi - -COPY . . - -ENV WATCHPACK_POLLING=true -ENV HOSTNAME="0.0.0.0" - -RUN chown -R node:node /app -USER node - -EXPOSE 3000 - -CMD ["sh", "-c", "if [ -f package-lock.json ]; then npm run dev; elif [ -f yarn.lock ]; then yarn dev; elif [ -f pnpm-lock.yaml ]; then pnpm dev; else npm run dev; fi"] -``` - -This file sets up a development environment for your Next.js app with hot module replacement and supports npm, yarn, and pnpm. - - -### Step 2: Update your `compose.yaml` file - -Open your `compose.yaml` file and define two services: one for production (`nextjs-prod-standalone`) and one for development (`nextjs-dev`). This matches the [sample project](https://github.com/kristiyan-velkov/docker-nextjs-sample) structure. - -Here's an example configuration for a Next.js application: - -```yaml -services: - nextjs-prod-standalone: - build: - context: . - dockerfile: Dockerfile - image: nextjs-sample:prod - container_name: nextjs-sample-prod - ports: - - "3000:3000" - - nextjs-dev: - build: - context: . - dockerfile: Dockerfile.dev - image: nextjs-sample:dev - container_name: nextjs-sample-dev - ports: - - "3000:3000" - environment: - - WATCHPACK_POLLING=true - develop: - watch: - - action: sync - path: . - target: /app - ignore: - - node_modules/ - - .next/ - - action: rebuild - path: package.json -``` - -- The `nextjs-prod-standalone` service builds and runs your production Next.js app using the standalone output. -- The `nextjs-dev` service runs your Next.js development server with hot module replacement. -- `watch` triggers file sync with Compose Watch. -- `WATCHPACK_POLLING=true` ensures file changes are detected properly inside Docker. -- The `rebuild` action for `package.json` ensures dependencies are reinstalled when the file changes. - -> [!NOTE] -> For more details, see the official guide: [Use Compose Watch](/manuals/compose/how-tos/file-watch.md). - -### Step 3: Configure Next.js for Docker development - -Next.js works well inside Docker containers out of the box, but there are a few configurations that can improve the development experience. - -The `next.config.ts` file you created during containerization already includes the `output: "standalone"` option for production. For development, Next.js automatically uses its built-in development server with hot reloading enabled. - -> [!NOTE] -> The Next.js development server automatically: -> - Enables Hot Module Replacement (HMR) for instant updates -> - Watches for file changes and recompiles automatically -> - Provides detailed error messages in the browser -> -> The `WATCHPACK_POLLING=true` environment variable in the compose file ensures file watching works correctly inside Docker containers. - - -After completing the previous steps, your project directory should now contain the following files: - -```text -├── docker-nextjs-sample/ -│ ├── Dockerfile -│ ├── Dockerfile.dev -│ ├── .dockerignore -│ ├── compose.yaml -│ └── next.config.ts -``` - -### Step 4: Start Compose Watch - -Run the following command from your project root to start your container in watch mode: - -```console -$ docker compose watch nextjs-dev -``` - -### Step 5: Test Compose Watch with Next.js - -To verify that Compose Watch is working correctly: - -1. Open the `app/page.tsx` file in your text editor (or `src/app/page.tsx` if your project uses a `src` directory). - -2. Locate the main content area and find a text element to modify. - -3. Make a visible change, for example, update a heading: - - ```tsx -

    Hello from Docker Compose Watch!

    - ``` - -4. Save the file. - -5. Open your browser at [http://localhost:3000](http://localhost:3000). - -You should see the updated text appear instantly, without needing to rebuild the container manually. This confirms that file watching and automatic synchronization are working as expected. - ---- - -## Summary - -In this section, you set up a complete development and production workflow for your Next.js application using Docker and Docker Compose. - -Here's what you achieved: -- Created a `Dockerfile.dev` to streamline local development with hot reloading -- Defined separate `nextjs-dev` and `nextjs-prod-standalone` services in your `compose.yaml` file -- Enabled real-time file syncing using Compose Watch for a smoother development experience -- Verified that live updates work seamlessly by modifying and previewing a component - -With this setup, you can build, run, and iterate on your Next.js app -entirely within containers across environments. - ---- - -## Related resources - -Deepen your knowledge and improve your containerized development workflow with these guides: - -- [Using Compose Watch](/manuals/compose/how-tos/file-watch.md) – Automatically sync source changes during development -- [Multi-stage builds](/manuals/build/building/multi-stage.md) – Create efficient, production-ready Docker images -- [Dockerfile best practices](/build/building/best-practices/) – Write clean, secure, and optimized Dockerfiles. -- [Compose file reference](/compose/compose-file/) – Learn the full syntax and options available for configuring services in `compose.yaml`. -- [Docker volumes](/storage/volumes/) – Persist and manage data between container runs - -## Next steps - -In the next section, you'll learn how to run unit tests for your Next.js application inside Docker containers. This ensures consistent testing across all environments and removes dependencies on local machine setup. diff --git a/content/guides/nextjs/run-tests.md b/content/guides/nextjs/run-tests.md deleted file mode 100644 index 69e486e76d38..000000000000 --- a/content/guides/nextjs/run-tests.md +++ /dev/null @@ -1,267 +0,0 @@ ---- -title: Run Next.js tests in a container -linkTitle: Run your tests and lint -weight: 40 -keywords: next.js, test, jest, vitest, lint, eslint -description: Learn how to run your Next.js tests and lint in a container. - ---- - -## Prerequisites - -Complete all the previous sections of this guide, starting with [Containerize Next.js application](containerize.md). - -## Overview - -Testing is a critical part of the development process. In this section, you'll learn how to: - -- Run unit tests using Vitest (or Jest) inside a Docker container. -- Run lint (e.g. ESLint) inside a Docker container. -- Use Docker Compose to run tests and lint in an isolated, reproducible environment. - -The [sample project](https://github.com/kristiyan-velkov/docker-nextjs-sample) uses [Vitest](https://vitest.dev/) with [Testing Library](https://testing-library.com/) for component testing. You can use the same setup or follow the alternative Jest configuration later. - ---- - -## Run tests during development - -The [sample project](https://github.com/kristiyan-velkov/docker-nextjs-sample) already includes lint (ESLint) and sample tests (Vitest, `app/page.test.tsx`) in place. If you're using the sample app, you can skip to **Step 3: Update compose.yaml** and run tests or lint with the commands below. If you're using your own project, follow the install and configuration steps to add the packages and scripts. - -The sample includes a test file at: - -```text -app/page.test.tsx -``` - -This file uses Vitest and React Testing Library to verify the behavior of page components. - -### Step 1: Install Vitest and React Testing Library (custom projects) - -If you're using a custom project and haven't already added the necessary testing tools, install them by running: - -```console -$ npm install --save-dev vitest @vitejs/plugin-react @testing-library/react @testing-library/dom jsdom -``` - -Then, update the scripts section of your `package.json` file to include: - -```json -"scripts": { - "test": "vitest", - "test:run": "vitest run" -} -``` - -For lint, add a `lint` script (and optionally `lint:fix`). For example, with [ESLint](https://eslint.org/): - -```json -"scripts": { - "test": "vitest", - "test:run": "vitest run", - "lint": "eslint .", - "lint:fix": "eslint . --fix" -} -``` - -The sample project uses `eslint` and `eslint-config-next` for Next.js. Install them in a custom project with: - -```console -$ npm install --save-dev eslint eslint-config-next @eslint/eslintrc -``` - -Create an ESLint config file (e.g. `eslint.config.cjs`) in your project root with Next.js rules and global ignores: - -```js -const { defineConfig, globalIgnores } = require("eslint/config"); -const { FlatCompat } = require("@eslint/eslintrc"); - -const compat = new FlatCompat({ baseDirectory: __dirname }); - -module.exports = defineConfig([ - ...compat.extends( - "eslint-config-next/core-web-vitals", - "eslint-config-next/typescript" - ), - globalIgnores([ - ".next/**", - "out/**", - "build/**", - "next-env.d.ts", - "node_modules/**", - "eslint.config.cjs", - ]), -]); -``` - ---- - -### Step 2: Configure Vitest (custom projects) - -If you're using a custom project, create a `vitest.config.ts` file in your project root (matching the [sample project](https://github.com/kristiyan-velkov/docker-nextjs-sample)): - -```ts -import { defineConfig } from "vitest/config"; -import react from "@vitejs/plugin-react"; - -export default defineConfig({ - plugins: [react()], - test: { - environment: "jsdom", - setupFiles: "./vitest.setup.ts", - globals: true, - }, -}); -``` - -Create a `vitest.setup.ts` file in your project root: - -```ts -import "@testing-library/jest-dom/vitest"; -``` - -> [!NOTE] -> Vitest works well with Next.js and provides fast execution and ESM support. For more details, see the [Next.js testing documentation](https://nextjs.org/docs/app/building-your-application/testing) and [Vitest docs](https://vitest.dev/). - -### Step 3: Update compose.yaml - -Add `nextjs-test` and `nextjs-lint` services to your `compose.yaml` file. In the sample project these services use the `tools` profile so they don't start with a normal `docker compose up`. Both reuse `Dockerfile.dev` and run the test or lint command: - -```yaml -services: - nextjs-prod-standalone: - build: - context: . - dockerfile: Dockerfile - image: nextjs-sample:prod - container_name: nextjs-sample-prod - ports: - - "3000:3000" - - nextjs-dev: - build: - context: . - dockerfile: Dockerfile.dev - image: nextjs-sample:dev - container_name: nextjs-sample-dev - ports: - - "3000:3000" - environment: - - WATCHPACK_POLLING=true - develop: - watch: - - action: sync - path: . - target: /app - ignore: - - node_modules/ - - .next/ - - action: rebuild - path: package.json - - nextjs-test: - build: - context: . - dockerfile: Dockerfile.dev - image: nextjs-sample:dev - container_name: nextjs-sample-test - command: - [ - "sh", - "-c", - "if [ -f package-lock.json ]; then npm run test:run 2>/dev/null || npm run test -- --run; elif [ -f yarn.lock ]; then yarn test:run 2>/dev/null || yarn test --run; elif [ -f pnpm-lock.yaml ]; then pnpm run test:run; else npm run test -- --run; fi", - ] - profiles: - - tools - - nextjs-lint: - build: - context: . - dockerfile: Dockerfile.dev - image: nextjs-sample:dev - container_name: nextjs-sample-lint - command: - [ - "sh", - "-c", - "if [ -f package-lock.json ]; then npm run lint; elif [ -f yarn.lock ]; then yarn lint; elif [ -f pnpm-lock.yaml ]; then pnpm lint; else npm run lint; fi", - ] - profiles: - - tools -``` - -The `nextjs-test` and `nextjs-lint` services reuse the same `Dockerfile.dev` used for [development](develop.md) and override the default command to run tests or lint. The `profiles: [tools]` means these services only run when you use the `--profile tools` option. - -After completing the previous steps, your project directory should contain: - -```text -├── docker-nextjs-sample/ -│ ├── Dockerfile -│ ├── Dockerfile.dev -│ ├── .dockerignore -│ ├── compose.yaml -│ ├── vitest.config.ts -│ ├── vitest.setup.ts -│ └── next.config.ts -``` - -### Step 4: Run the tests - -To execute your test suite inside the container, run from your project root: - -```console -$ docker compose --profile tools run --rm nextjs-test -``` - -This command will: -- Start the `nextjs-test` service (because of `--profile tools`). -- Run your test script (`test:run` or `test -- --run`) in the same environment as development. -- Remove the container after the tests complete ([`docker compose run --rm`](/reference/cli/docker/compose/run/)). - -> [!NOTE] -> For more information about Compose commands and profiles, see the [Compose CLI reference](/reference/cli/docker/compose/). - -### Step 5: Run lint in the container - -To run your linter (e.g. ESLint) inside the container, use the `nextjs-lint` service with the same `tools` profile: - -```console -$ docker compose --profile tools run --rm nextjs-lint -``` - -This command will: -- Start the `nextjs-lint` service (because of `--profile tools`). -- Run your lint script (`npm run lint`, `yarn lint`, or `pnpm lint` depending on your lockfile) in the same environment as development. -- Remove the container after lint completes. - -Ensure your `package.json` includes a `lint` script. The sample project already has `"lint": "eslint ."` and `"lint:fix": "eslint . --fix"`; for a custom project, add the same and install `eslint` and `eslint-config-next` if needed. - ---- - -## Summary - -In this section, you learned how to run unit tests for your Next.js application inside a Docker container using Vitest and Docker Compose. - -What you accomplished: -- Installed and configured Vitest and React Testing Library for testing Next.js components. -- Created `nextjs-test` and `nextjs-lint` services in `compose.yaml` (with `tools` profile) to isolate test and lint execution. -- Reused the development `Dockerfile.dev` to ensure consistency between dev, test, and lint environments. -- Ran tests inside the container using `docker compose --profile tools run --rm nextjs-test`. -- Ran lint inside the container using `docker compose --profile tools run --rm nextjs-lint`. -- Ensured reliable, repeatable testing and linting across environments without relying on local machine setup. - ---- - -## Related resources - -Explore official references and best practices to sharpen your Docker testing workflow: - -- [Dockerfile reference](/reference/dockerfile/) – Understand all Dockerfile instructions and syntax. -- [Best practices for writing Dockerfiles](/develop/develop-images/dockerfile_best-practices/) – Write efficient, maintainable, and secure Dockerfiles. -- [Compose file reference](/compose/compose-file/) – Learn the full syntax and options available for configuring services in `compose.yaml`. -- [`docker compose run` CLI reference](/reference/cli/docker/compose/run/) – Run one-off commands in a service container. -- [Next.js Testing Documentation](https://nextjs.org/docs/app/building-your-application/testing) – Official Next.js testing guide. ---- - -## Next steps - -Next, you'll learn how to set up a CI/CD pipeline using GitHub Actions to automatically build and test your Next.js application in a containerized environment. This ensures your code is validated on every push or pull request, maintaining consistency and reliability across your development workflow. diff --git a/content/guides/opencode-model-runner.md b/content/guides/opencode-model-runner.md index e02d5d523b1e..fa79a9d8b0db 100644 --- a/content/guides/opencode-model-runner.md +++ b/content/guides/opencode-model-runner.md @@ -5,8 +5,8 @@ summary: | Connect OpenCode to Docker Model Runner with an OpenAI-compatible endpoint, choose coding models, and package `gpt-oss` with a larger context window. keywords: ai, opencode, docker model runner, local models, coding assistant -tags: [ai] params: + tags: [ai] time: 10 minutes --- diff --git a/content/guides/opentelemetry.md b/content/guides/opentelemetry.md index 6c063391c363..8b9bf4bbfbef 100644 --- a/content/guides/opentelemetry.md +++ b/content/guides/opentelemetry.md @@ -4,9 +4,8 @@ description: &desc Learn how to instrument a JavaScript application using OpenTe keywords: OpenTelemetry, observability, tracing linktitle: Instrumenting JS Apps with OpenTelemetry summary: *desc -tags: [app-dev, observability] -languages: [js] params: + tags: [cicd] time: 10 minutes --- diff --git a/content/guides/orchestration.md b/content/guides/orchestration.md index 8127e98e611a..26c4c56d54c1 100644 --- a/content/guides/orchestration.md +++ b/content/guides/orchestration.md @@ -7,8 +7,8 @@ aliases: - /guides/deployment-orchestration/orchestration/ summary: | Explore the essentials of container orchestration with Docker. -tags: [deploy] params: + tags: [deployment] time: 10 minutes --- diff --git a/content/guides/pgadmin.md b/content/guides/pgadmin.md index e7cddb69363e..2a688bbb181e 100644 --- a/content/guides/pgadmin.md +++ b/content/guides/pgadmin.md @@ -5,8 +5,8 @@ title: Visualizing your PostgreSQL databases with pgAdmin linktitle: Visualizing your PostgreSQL databases with pgAdmin summary: | Explore how to add pgAdmin to your development stack and make it as easy as possible for your teammates to navigate through your PostgreSQL databases. -tags: [databases] params: + tags: [databases] time: 10 minutes --- diff --git a/content/guides/php/_index.md b/content/guides/php/_index.md index 358b13865aa9..394197735138 100644 --- a/content/guides/php/_index.md +++ b/content/guides/php/_index.md @@ -5,16 +5,25 @@ description: Containerize and develop PHP apps using Docker keywords: getting started, php, composer summary: | This guide explains how to containerize PHP applications using Docker. -toc_min: 1 -toc_max: 2 aliases: - /language/php/ - /guides/language/php/ -languages: [php] + - /language/php/containerize/ + - /language/php/develop/ + - /language/php/run-tests/ + - /language/php/configure-ci-cd/ + - /language/php/deploy/ + - /guides/php/configure-ci-cd/ + - /guides/php/containerize/ + - /guides/php/deploy/ + - /guides/php/develop/ + - /guides/php/run-tests/ params: + tags: [cicd] time: 20 minutes --- + The PHP language-specific guide teaches you how to create a containerized PHP application using Docker. In this guide, you'll learn how to: - Containerize and run a PHP application @@ -26,3 +35,1116 @@ The PHP language-specific guide teaches you how to create a containerized PHP ap After completing the PHP language-specific guide, you should be able to containerize your own PHP application based on the examples and instructions provided in this guide. Start by containerizing an existing PHP application. + +## Containerize a PHP application + +### Prerequisites + +- You have installed the latest version of [Docker + Desktop](/get-started/get-docker.md). +- You have a [git client](https://git-scm.com/downloads). The examples in this + section use a command-line based git client, but you can use any client. + +### Overview + +This section walks you through containerizing and running a PHP +application. + +### Get the sample applications + +In this guide, you will use a pre-built PHP application. The application uses Composer for library dependency management. You'll serve the application via an Apache web server. + +Open a terminal, change directory to a directory that you want to work in, and +run the following command to clone the repository. + +```console +$ git clone https://github.com/docker/docker-php-sample +``` + +The sample application is a basic hello world application and an application that increments a counter in a database. In addition, the application uses PHPUnit for testing. + +### Create Docker assets + +Now that you have an application, you can create the necessary Docker assets to +containerize it. + +> [!TIP] +> +> [Gordon](/ai/gordon/), Docker's AI assistant, can generate Docker assets for your project. Ask Gordon to create a Dockerfile, Compose file, and `.dockerignore` tailored to your application. + +Create the following files in your `docker-php-sample` directory. + +```dockerfile {collapse=true,title=Dockerfile} +# syntax=docker/dockerfile:1 + +# Comments are provided throughout this file to help you get started. +# If you need more help, visit the Dockerfile reference guide at +# https://docs.docker.com/go/dockerfile-reference/ + +################################################################################ + +# Create a stage for installing app dependencies defined in Composer. +FROM composer:lts as deps + +WORKDIR /app + +# If your composer.json file defines scripts that run during dependency installation and +# reference your application source files, uncomment the line below to copy all the files +# into this layer. +# COPY . . + +# Download dependencies as a separate step to take advantage of Docker's caching. +# Leverage a bind mounts to composer.json and composer.lock to avoid having to copy them +# into this layer. +# Leverage a cache mount to /tmp/cache so that subsequent builds don't have to re-download packages. +RUN --mount=type=bind,source=composer.json,target=composer.json \ + --mount=type=bind,source=composer.lock,target=composer.lock \ + --mount=type=cache,target=/tmp/cache \ + composer install --no-dev --no-interaction + +################################################################################ + +# Create a new stage for running the application that contains the minimal +# runtime dependencies for the application. This often uses a different base +# image from the install or build stage where the necessary files are copied +# from the install stage. +# +# The example below uses the PHP Apache image as the foundation for running the app. +# By specifying the "8.2-apache" tag, it will also use whatever happens to be the +# most recent version of that tag when you build your Dockerfile. +# If reproducibility is important, consider using a specific digest SHA, like +# php@sha256:99cede493dfd88720b610eb8077c8688d3cca50003d76d1d539b0efc8cca72b4. +FROM php:8.2-apache as final + +# Your PHP application may require additional PHP extensions to be installed +# manually. For detailed instructions for installing extensions can be found, see +# https://github.com/docker-library/docs/tree/master/php#how-to-install-more-php-extensions +# The following code blocks provide examples that you can edit and use. +# +# Add core PHP extensions, see +# https://github.com/docker-library/docs/tree/master/php#php-core-extensions +# This example adds the apt packages for the 'gd' extension's dependencies and then +# installs the 'gd' extension. For additional tips on running apt-get, see +# https://docs.docker.com/go/dockerfile-aptget-best-practices/ +# RUN apt-get update && apt-get install -y \ +# libfreetype-dev \ +# libjpeg62-turbo-dev \ +# libpng-dev \ +# && rm -rf /var/lib/apt/lists/* \ +# && docker-php-ext-configure gd --with-freetype --with-jpeg \ +# && docker-php-ext-install -j$(nproc) gd +# +# Add PECL extensions, see +# https://github.com/docker-library/docs/tree/master/php#pecl-extensions +# This example adds the 'redis' and 'xdebug' extensions. +# RUN pecl install redis-5.3.7 \ +# && pecl install xdebug-3.2.1 \ +# && docker-php-ext-enable redis xdebug + +# Use the default production configuration for PHP runtime arguments, see +# https://github.com/docker-library/docs/tree/master/php#configuration +RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini" + +# Copy the app dependencies from the previous install stage. +COPY --from=deps app/vendor/ /var/www/html/vendor +# Copy the app files from the app directory. +COPY ./src /var/www/html + +# Switch to a non-privileged user (defined in the base image) that the app will run under. +# See https://docs.docker.com/go/dockerfile-user-best-practices/ +USER www-data +``` + +```yaml {collapse=true,title=compose.yaml} +# Comments are provided throughout this file to help you get started. +# If you need more help, visit the Docker Compose reference guide at +# https://docs.docker.com/go/compose-spec-reference/ + +# Here the instructions define your application as a service called "server". +# This service is built from the Dockerfile in the current directory. +# You can add other services your application may depend on here, such as a +# database or a cache. For examples, see the Awesome Compose repository: +# https://github.com/docker/awesome-compose +services: + server: + build: + context: . + ports: + - 9000:80 + +# The commented out section below is an example of how to define a PostgreSQL +# database that your application can use. `depends_on` tells Docker Compose to +# start the database before your application. The `db-data` volume persists the +# database data between container restarts. The `db-password` secret is used +# to set the database password. You must create `db/password.txt` and add +# a password of your choosing to it before running `docker compose up`. +# depends_on: +# db: +# condition: service_healthy +# db: +# image: postgres +# restart: always +# user: postgres +# secrets: +# - db-password +# volumes: +# - db-data:/var/lib/postgresql/data +# environment: +# - POSTGRES_DB=example +# - POSTGRES_PASSWORD_FILE=/run/secrets/db-password +# expose: +# - 5432 +# healthcheck: +# test: [ "CMD", "pg_isready" ] +# interval: 10s +# timeout: 5s +# retries: 5 +# volumes: +# db-data: +# secrets: +# db-password: +# file: db/password.txt +``` + +```text {collapse=true,title=".dockerignore"} +# Include any files or directories that you don't want to be copied to your +# container here (e.g., local build artifacts, temporary files, etc.). +# +# For more help, visit the .dockerignore file reference guide at +# https://docs.docker.com/go/build-context-dockerignore/ + +**/.classpath +**/.dockerignore +**/.env +**/.git +**/.gitignore +**/.project +**/.settings +**/.toolstarget +**/.vs +**/.vscode +**/.next +**/.cache +**/*.*proj.user +**/*.dbmdl +**/*.jfm +**/charts +**/docker-compose* +**/compose.y*ml +!**/composer.json +!**/composer.lock +**/Dockerfile* +**/node_modules +**/npm-debug.log +**/obj +**/secrets.dev.yaml +**/values.dev.yaml +**/vendor +LICENSE +README.md +``` + +You should now have the following contents in your `docker-php-sample` +directory. + +```text +├── docker-php-sample/ +│ ├── .git/ +│ ├── src/ +│ ├── tests/ +│ ├── .dockerignore +│ ├── .gitignore +│ ├── compose.yaml +│ ├── composer.json +│ ├── composer.lock +│ ├── Dockerfile +│ └── README.md +``` + +To learn more about these files, see the following: + +- [Dockerfile](/reference/dockerfile.md) +- [.dockerignore](/reference/dockerfile.md#dockerignore-file) +- [compose.yaml](/reference/compose-file/_index.md) + +### Run the application + +Inside the `docker-php-sample` directory, run the following command in a +terminal. + +```console +$ docker compose up --build +``` + +Open a browser and view the application at [http://localhost:9000/hello.php](http://localhost:9000/hello.php). You should see a simple hello world application. + +In the terminal, press `ctrl`+`c` to stop the application. + +#### Run the application in the background + +You can run the application detached from the terminal by adding the `-d` +option. Inside the `docker-php-sample` directory, run the following command +in a terminal. + +```console +$ docker compose up --build -d +``` + +Open a browser and view the application at [http://localhost:9000/hello.php](http://localhost:9000/hello.php). You should see a simple hello world application. + +In the terminal, run the following command to stop the application. + +```console +$ docker compose down +``` + +For more information about Compose commands, see the [Compose CLI +reference](/reference/cli/docker/compose/). + +### Summary + +In this section, you learned how you can containerize and run a simple PHP +application using Docker. + +### Next steps + +In the next section, you'll learn how you can develop your application using +Docker containers. + +## Use containers for PHP development + +### Prerequisites + +Complete [Containerize a PHP application](containerize.md). + +### Overview + +In this section, you'll learn how to set up a development environment for your containerized application. This includes: + +- Adding a local database and persisting data +- Adding phpMyAdmin to interact with the database +- Configuring Compose to automatically update your running Compose services as + you edit and save your code +- Creating a development container that contains the dev dependencies + +### Add a local database and persist data + +You can use containers to set up local services, like a database. +To do this for the sample application, you'll need to do the following: + +- Update the `Dockerfile` to install extensions to connect to the database +- Update the `compose.yaml` file to add a database service and volume to persist data + +#### Update the Dockerfile to install extensions + +To install PHP extensions, you need to update the `Dockerfile`. Open your +Dockerfile in an IDE or text editor and then update the contents. The following +`Dockerfile` includes one new line that installs the `pdo` and `pdo_mysql` +extensions. All comments have been removed. + +```dockerfile {hl_lines=11} +# syntax=docker/dockerfile:1 + +FROM composer:lts as deps +WORKDIR /app +RUN --mount=type=bind,source=composer.json,target=composer.json \ + --mount=type=bind,source=composer.lock,target=composer.lock \ + --mount=type=cache,target=/tmp/cache \ + composer install --no-dev --no-interaction + +FROM php:8.2-apache as final +RUN docker-php-ext-install pdo pdo_mysql +RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini" +COPY --from=deps app/vendor/ /var/www/html/vendor +COPY ./src /var/www/html +USER www-data +``` + +For more details about installing PHP extensions, see the [Official Docker Image for PHP](https://hub.docker.com/_/php). + +#### Update the compose.yaml file to add a db and persist data + +Open the `compose.yaml` file in an IDE or text editor. You'll notice it +already contains commented-out instructions for a PostgreSQL database and volume. For this application, you'll use MariaDB. For more details about MariaDB, see the [MariaDB Official Docker image](https://hub.docker.com/_/mariadb). + +Open the `src/database.php` file in an IDE or text editor. You'll notice that it reads environment variables in order to connect to the database. + +In the `compose.yaml` file, you'll need to update the following: + +1. Uncomment and update the database instructions for MariaDB. +2. Add a secret to the server service to pass in the database password. +3. Add the database connection environment variables to the server service. +4. Uncomment the volume instructions to persist data. + +The following is the updated `compose.yaml` file. All comments have been removed. + +```yaml +services: + server: + build: + context: . + ports: + - 9000:80 + depends_on: + db: + condition: service_healthy + secrets: + - db-password + environment: + - PASSWORD_FILE_PATH=/run/secrets/db-password + - DB_HOST=db + - DB_NAME=example + - DB_USER=root + db: + image: mariadb + restart: always + user: root + secrets: + - db-password + volumes: + - db-data:/var/lib/mysql + environment: + - MARIADB_ROOT_PASSWORD_FILE=/run/secrets/db-password + - MARIADB_DATABASE=example + expose: + - 3306 + healthcheck: + test: + [ + "CMD", + "/usr/local/bin/healthcheck.sh", + "--su-mysql", + "--connect", + "--innodb_initialized", + ] + interval: 10s + timeout: 5s + retries: 5 +volumes: + db-data: +secrets: + db-password: + file: db/password.txt +``` + +> [!NOTE] +> +> To learn more about the instructions in the Compose file, see [Compose file +> reference](/reference/compose-file/). + +Before you run the application using Compose, notice that this Compose file uses +`secrets` and specifies a `password.txt` file to hold the database's password. +You must create this file as it's not included in the source repository. + +In the `docker-php-sample` directory, create a new directory named `db` and +inside that directory create a file named `password.txt`. Open `password.txt` in an IDE or text editor and add the following password. The password must be on a single line, with no additional lines in the file. + +```text +example +``` + +Save and close the `password.txt` file. + +You should now have the following in your `docker-php-sample` directory. + +```text +├── docker-php-sample/ +│ ├── .git/ +│ ├── db/ +│ │ └── password.txt +│ ├── src/ +│ ├── tests/ +│ ├── .dockerignore +│ ├── .gitignore +│ ├── compose.yaml +│ ├── composer.json +│ ├── composer.lock +│ ├── Dockerfile +│ └── README.md +``` + +Run the following command to start your application. + +```console +$ docker compose up --build +``` + +Open a browser and view the application at [http://localhost:9000/database.php](http://localhost:9000/database.php). You should see a simple web application with text and a counter that increments every time you refresh. + +Press `ctrl+c` in the terminal to stop your application. + +### Verify that data persists in the database + +In the terminal, run `docker compose rm` to remove your containers and then run `docker compose up` to run your application again. + +```console +$ docker compose rm +$ docker compose up --build +``` + +Refresh [http://localhost:9000/database.php](http://localhost:9000/database.php) in your browser and verify that the previous count still exists. Without a volume, the database data wouldn't persist after you remove the container. + +Press `ctrl+c` in the terminal to stop your application. + +### Add phpMyAdmin to interact with the database + +You can easily add services to your application stack by updating the `compose.yaml` file. + +Update your `compose.yaml` to add a new service for phpMyAdmin. For more details, see the [phpMyAdmin Official Docker Image](https://hub.docker.com/_/phpmyadmin). The following is the updated `compose.yaml` file. + +```yaml {hl_lines="42-49"} +services: + server: + build: + context: . + ports: + - 9000:80 + depends_on: + db: + condition: service_healthy + secrets: + - db-password + environment: + - PASSWORD_FILE_PATH=/run/secrets/db-password + - DB_HOST=db + - DB_NAME=example + - DB_USER=root + db: + image: mariadb + restart: always + user: root + secrets: + - db-password + volumes: + - db-data:/var/lib/mysql + environment: + - MARIADB_ROOT_PASSWORD_FILE=/run/secrets/db-password + - MARIADB_DATABASE=example + expose: + - 3306 + healthcheck: + test: + [ + "CMD", + "/usr/local/bin/healthcheck.sh", + "--su-mysql", + "--connect", + "--innodb_initialized", + ] + interval: 10s + timeout: 5s + retries: 5 + phpmyadmin: + image: phpmyadmin + ports: + - 8080:80 + depends_on: + - db + environment: + - PMA_HOST=db +volumes: + db-data: +secrets: + db-password: + file: db/password.txt +``` + +In the terminal, run `docker compose up` to run your application again. + +```console +$ docker compose up --build +``` + +Open [http://localhost:8080](http://localhost:8080) in your browser to access phpMyAdmin. Log in using `root` as the username and `example` as the password. You can now interact with the database through phpMyAdmin. + +Press `ctrl+c` in the terminal to stop your application. + +### Automatically update services + +Use Compose Watch to automatically update your running Compose services as you edit and save your code. For more details about Compose Watch, see [Use Compose Watch](/manuals/compose/how-tos/file-watch.md). + +Open your `compose.yaml` file in an IDE or text editor and then add the Compose Watch instructions. The following is the updated `compose.yaml` file. + +```yaml {hl_lines="17-21"} +services: + server: + build: + context: . + ports: + - 9000:80 + depends_on: + db: + condition: service_healthy + secrets: + - db-password + environment: + - PASSWORD_FILE_PATH=/run/secrets/db-password + - DB_HOST=db + - DB_NAME=example + - DB_USER=root + develop: + watch: + - action: sync + path: ./src + target: /var/www/html + db: + image: mariadb + restart: always + user: root + secrets: + - db-password + volumes: + - db-data:/var/lib/mysql + environment: + - MARIADB_ROOT_PASSWORD_FILE=/run/secrets/db-password + - MARIADB_DATABASE=example + expose: + - 3306 + healthcheck: + test: + [ + "CMD", + "/usr/local/bin/healthcheck.sh", + "--su-mysql", + "--connect", + "--innodb_initialized", + ] + interval: 10s + timeout: 5s + retries: 5 + phpmyadmin: + image: phpmyadmin + ports: + - 8080:80 + depends_on: + - db + environment: + - PMA_HOST=db +volumes: + db-data: +secrets: + db-password: + file: db/password.txt +``` + +Run the following command to run your application with Compose Watch. + +```console +$ docker compose watch +``` + +Open a browser and verify that the application is running at [http://localhost:9000/hello.php](http://localhost:9000/hello.php). + +Any changes to the application's source files on your local machine will now be +immediately reflected in the running container. + +Open `hello.php` in an IDE or text editor and update the string `Hello, world!` to `Hello, Docker!`. + +Save the changes to `hello.php` and then wait a few seconds for the application to sync. Refresh [http://localhost:9000/hello.php](http://localhost:9000/hello.php) in your browser and verify that the updated text appears. + +Press `ctrl+c` in the terminal to stop Compose Watch. Run `docker compose down` in the terminal to stop the application. + +### Create a development container + +At this point, when you run your containerized application, Composer isn't installing the dev dependencies. While this small image is good for production, it lacks the tools and dependencies you may need when developing and it doesn't include the `tests` directory. You can use multi-stage builds to build stages for both development and production in the same Dockerfile. For more details, see [Multi-stage builds](/manuals/build/building/multi-stage.md). + +In the `Dockerfile`, you'll need to update the following: + +1. Split the `deps` staged into two stages. One stage for production + (`prod-deps`) and one stage (`dev-deps`) to install development dependencies. +2. Create a common `base` stage. +3. Create a new `development` stage for development. +4. Update the `final` stage to copy dependencies from the new `prod-deps` stage. + +The following is the `Dockerfile` before and after the changes. + +{{< tabs >}} +{{< tab name="Before" >}} + +```dockerfile +# syntax=docker/dockerfile:1 + +FROM composer:lts as deps +WORKDIR /app +RUN --mount=type=bind,source=composer.json,target=composer.json \ + --mount=type=bind,source=composer.lock,target=composer.lock \ + --mount=type=cache,target=/tmp/cache \ + composer install --no-dev --no-interaction + +FROM php:8.2-apache as final +RUN docker-php-ext-install pdo pdo_mysql +RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini" +COPY --from=deps app/vendor/ /var/www/html/vendor +COPY ./src /var/www/html +USER www-data +``` + +{{< /tab >}} +{{< tab name="After" >}} + +```dockerfile +# syntax=docker/dockerfile:1 + +FROM composer:lts as prod-deps +WORKDIR /app +RUN --mount=type=bind,source=./composer.json,target=composer.json \ + --mount=type=bind,source=./composer.lock,target=composer.lock \ + --mount=type=cache,target=/tmp/cache \ + composer install --no-dev --no-interaction + +FROM composer:lts as dev-deps +WORKDIR /app +RUN --mount=type=bind,source=./composer.json,target=composer.json \ + --mount=type=bind,source=./composer.lock,target=composer.lock \ + --mount=type=cache,target=/tmp/cache \ + composer install --no-interaction + +FROM php:8.2-apache as base +RUN docker-php-ext-install pdo pdo_mysql +COPY ./src /var/www/html + +FROM base as development +COPY ./tests /var/www/html/tests +RUN mv "$PHP_INI_DIR/php.ini-development" "$PHP_INI_DIR/php.ini" +COPY --from=dev-deps app/vendor/ /var/www/html/vendor + +FROM base as final +RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini" +COPY --from=prod-deps app/vendor/ /var/www/html/vendor +USER www-data +``` + +{{< /tab >}} +{{< /tabs >}} + +Update your `compose.yaml` file by adding an instruction to target the +development stage. + +The following is the updated section of the `compose.yaml` file. + +```yaml {hl_lines=5} +services: + server: + build: + context: . + target: development + # ... +``` + +Your containerized application will now install the dev dependencies. + +Run the following command to start your application. + +```console +$ docker compose up --build +``` + +Open a browser and view the application at [http://localhost:9000/hello.php](http://localhost:9000/hello.php). You should still see the simple "Hello, Docker!" application. + +Press `ctrl+c` in the terminal to stop your application. + +While the application appears the same, you can now make use of the dev dependencies. Continue to the next section to learn how you can run tests using Docker. + +### Summary + +In this section, you took a look at setting up your Compose file to add a local +database and persist data. You also learned how to use Compose Watch to automatically sync your application when you update your code. And finally, you learned how to create a development container that contains the dependencies needed for development. + +Related information: + +- [Compose file reference](/reference/compose-file/) +- [Compose file watch](/manuals/compose/how-tos/file-watch.md) +- [Dockerfile reference](/reference/dockerfile.md) +- [Official Docker Image for PHP](https://hub.docker.com/_/php) + +### Next steps + +In the next section, you'll learn how to run unit tests using Docker. + +## Run PHP tests in a container + +### Prerequisites + +Complete all the previous sections of this guide, starting with [Containerize a PHP application](containerize.md). + +### Overview + +Testing is an essential part of modern software development. Testing can mean a +lot of things to different development teams. There are unit tests, integration +tests and end-to-end testing. In this guide you take a look at running your unit +tests in Docker when developing and when building. + +### Run tests when developing locally + +The sample application already has a PHPUnit test inside the `tests` directory. When developing locally, you can use Compose to run your tests. + +Run the following command in the `docker-php-sample` directory to run the tests inside a container. + +```console +$ docker compose run --build --rm server ./vendor/bin/phpunit tests/HelloWorldTest.php +``` + +You should see output that contains the following. + +```console +Hello, Docker!PHPUnit 9.6.13 by Sebastian Bergmann and contributors. + +. 1 / 1 (100%) + +Time: 00:00.003, Memory: 4.00 MB + +OK (1 test, 1 assertion) +``` + +To learn more about the command, see [docker compose run](/reference/cli/docker/compose/run/). + +### Run tests when building + +To run your tests when building, you need to update your Dockerfile. Create a new test stage that runs the tests. + +The following is the updated Dockerfile. + +```dockerfile {hl_lines="26-28"} +# syntax=docker/dockerfile:1 + +FROM composer:lts as prod-deps +WORKDIR /app +RUN --mount=type=bind,source=./composer.json,target=composer.json \ + --mount=type=bind,source=./composer.lock,target=composer.lock \ + --mount=type=cache,target=/tmp/cache \ + composer install --no-dev --no-interaction + +FROM composer:lts as dev-deps +WORKDIR /app +RUN --mount=type=bind,source=./composer.json,target=composer.json \ + --mount=type=bind,source=./composer.lock,target=composer.lock \ + --mount=type=cache,target=/tmp/cache \ + composer install --no-interaction + +FROM php:8.2-apache as base +RUN docker-php-ext-install pdo pdo_mysql +COPY ./src /var/www/html + +FROM base as development +COPY ./tests /var/www/html/tests +RUN mv "$PHP_INI_DIR/php.ini-development" "$PHP_INI_DIR/php.ini" +COPY --from=dev-deps app/vendor/ /var/www/html/vendor + +FROM development as test +WORKDIR /var/www/html +RUN ./vendor/bin/phpunit tests/HelloWorldTest.php + +FROM base as final +RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini" +COPY --from=prod-deps app/vendor/ /var/www/html/vendor +USER www-data +``` + +Run the following command to build an image using the test stage as the target and view the test results. Include `--progress plain` to view the build output, `--no-cache` to ensure the tests always run, and `--target test` to target the test stage. + +```console +$ docker build -t php-docker-image-test --progress plain --no-cache --target test . +``` + +You should see output containing the following. + +```console +#18 [test 2/2] RUN ./vendor/bin/phpunit tests/HelloWorldTest.php +#18 0.385 Hello, Docker!PHPUnit 9.6.13 by Sebastian Bergmann and contributors. +#18 0.392 +#18 0.394 . 1 / 1 (100%) +#18 0.395 +#18 0.395 Time: 00:00.003, Memory: 4.00 MB +#18 0.395 +#18 0.395 OK (1 test, 1 assertion) +``` + +### Summary + +In this section, you learned how to run tests when developing locally using Compose and how to run tests when building your image. + +Related information: + +- [docker compose run](/reference/cli/docker/compose/run/) + +### Next steps + +Next, you’ll learn how to set up a CI/CD pipeline using GitHub Actions. + +## Configure CI/CD for your PHP application + +### Prerequisites + +Complete all the previous sections of this guide, starting with [Containerize a PHP application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. + +### Overview + +In this section, you'll learn how to set up and use GitHub Actions to build and test your Docker image as well as push it to Docker Hub. You will complete the following steps: + +1. Create a new repository on GitHub. +2. Define the GitHub Actions workflow. +3. Run the workflow. + +### Step one: Create the repository + +Create a GitHub repository, configure the Docker Hub credentials, and push your source code. + +1. [Create a new repository](https://github.com/new) on GitHub. + +2. Open the repository **Settings**, and go to **Secrets and variables** > + **Actions**. + +3. Create a new **Repository variable** named `DOCKER_USERNAME` and your Docker ID as a value. + +4. Create a new [Personal Access Token (PAT)](/manuals/security/access-tokens.md#create-an-access-token) for Docker Hub. You can name this token `docker-tutorial`. Make sure access permissions include Read and Write. + +5. Add the PAT as a **Repository secret** in your GitHub repository, with the name + `DOCKERHUB_TOKEN`. + +6. In your local repository on your machine, run the following command to change + the origin to the repository you just created. Make sure you change + `your-username` to your GitHub username and `your-repository` to the name of + the repository you created. + + ```console + $ git remote set-url origin https://github.com/your-username/your-repository.git + ``` + +7. In your local repository on your machine, run the following command to rename + the branch to main. + + ```console + $ git branch -M main + ``` + +8. Run the following commands to stage, commit, and then push your local + repository to GitHub. + + ```console + $ git add -A + $ git commit -m "my first commit" + $ git push -u origin main + ``` + +### Step two: Set up the workflow + +Set up your GitHub Actions workflow for building, testing, and pushing the image +to Docker Hub. + +1. Go to your repository on GitHub and then select the **Actions** tab. + +2. Select **set up a workflow yourself**. + + This takes you to a page for creating a new GitHub actions workflow file in + your repository, under `.github/workflows/main.yml` by default. + +3. In the editor window, copy and paste the following YAML configuration. + + ```yaml + name: ci + + on: + push: + branches: + - main + + jobs: + build: + runs-on: ubuntu-latest + steps: + - name: Login to Docker Hub + uses: docker/login-action@{{% param "login_action_version" %}} + with: + username: ${{ vars.DOCKER_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} + + - name: Build and test + uses: docker/build-push-action@{{% param "build_push_action_version" %}} + with: + target: test + load: true + + - name: Build and push + uses: docker/build-push-action@{{% param "build_push_action_version" %}} + with: + platforms: linux/amd64,linux/arm64 + push: true + target: final + tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest + ``` + + For more information about the YAML syntax for `docker/build-push-action`, + refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). + +### Step three: Run the workflow + +Save the workflow file and run the job. + +1. Select **Commit changes...** and push the changes to the `main` branch. + + After pushing the commit, the workflow starts automatically. + +2. Go to the **Actions** tab. It displays the workflow. + + Selecting the workflow shows you the breakdown of all the steps. + +3. When the workflow is complete, go to your + [repositories on Docker Hub](https://hub.docker.com/repositories). + + If you see the new repository in that list, it means the GitHub Actions + successfully pushed the image to Docker Hub. + +### Summary + +In this section, you learned how to set up a GitHub Actions workflow for your application. + +Related information: + +- [Introduction to GitHub Actions](/guides/gha.md) +- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) +- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) + +### Next steps + +Next, learn how you can locally test and debug your workloads on Kubernetes before deploying. + +## Test your PHP deployment + +### Prerequisites + +- Complete all the previous sections of this guide, starting with [Containerize + a PHP application](containerize.md). +- [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker + Desktop. + +### Overview + +In this section, you'll learn how to use Docker Desktop to deploy your +application to a fully-featured Kubernetes environment on your development +machine. This allows you to test and debug your workloads on Kubernetes locally +before deploying. + +### Create a Kubernetes YAML file + +In your `docker-php-sample` directory, create a file named +`docker-php-kubernetes.yaml`. Open the file in an IDE or text editor and add +the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker +username and the name of the repository that you created in [Configure CI/CD for +your PHP application](configure-ci-cd.md). + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: docker-php-demo + namespace: default +spec: + replicas: 1 + selector: + matchLabels: + hello-php: web + template: + metadata: + labels: + hello-php: web + spec: + containers: + - name: hello-site + image: DOCKER_USERNAME/REPO_NAME + imagePullPolicy: Always +--- +apiVersion: v1 +kind: Service +metadata: + name: php-entrypoint + namespace: default +spec: + type: NodePort + selector: + hello-php: web + ports: + - port: 80 + targetPort: 80 + nodePort: 30001 +``` + +In this Kubernetes YAML file, there are two objects, separated by the `---`: + +- A Deployment, describing a scalable group of identical pods. In this case, + you'll get just one replica, or copy of your pod. That pod, which is + described under `template`, has just one container in it. The container is + created from the image built by GitHub Actions in [Configure CI/CD for your + PHP application](configure-ci-cd.md). +- A NodePort service, which will route traffic from port 30001 on your host to + port 80 inside the pods it routes to, allowing you to reach your app + from the network. + +To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). + +### Deploy and check your application + +1. In a terminal, navigate to the `docker-php-sample` directory + and deploy your application to Kubernetes. + + ```console + $ kubectl apply -f docker-php-kubernetes.yaml + ``` + + You should see output that looks like the following, indicating your Kubernetes objects were created successfully. + + ```text + deployment.apps/docker-php-demo created + service/php-entrypoint created + ``` + +2. Make sure everything worked by listing your deployments. + + ```console + $ kubectl get deployments + ``` + + Your deployment should be listed as follows: + + ```text + NAME READY UP-TO-DATE AVAILABLE AGE + docker-php-demo 1/1 1 1 6s + ``` + + This indicates all of the pods are up and running. Do the same check for your services. + + ```console + $ kubectl get services + ``` + + You should get output like the following. + + ```text + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + kubernetes ClusterIP 10.96.0.1 443/TCP 7d22h + php-entrypoint NodePort 10.111.101.229 80:30001/TCP 33s + ``` + + In addition to the default `kubernetes` service, you can see your `php-entrypoint` service. The `php-entrypoint` service is accepting traffic on port 30001/TCP. + +3. Open a browser and visit your app at + [http://localhost:30001/hello.php](http://localhost:30001/hello.php). You + should see your application. + +4. Run the following command to tear down your application. + + ```console + $ kubectl delete -f docker-php-kubernetes.yaml + ``` + +### Summary + +In this section, you learned how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. + +Related information: + +- [Kubernetes documentation](https://kubernetes.io/docs/home/) +- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) +- [Swarm mode overview](/manuals/engine/swarm/_index.md) diff --git a/content/guides/php/configure-ci-cd.md b/content/guides/php/configure-ci-cd.md deleted file mode 100644 index 3f1053e5b870..000000000000 --- a/content/guides/php/configure-ci-cd.md +++ /dev/null @@ -1,148 +0,0 @@ ---- -title: Configure CI/CD for your PHP application -linkTitle: Configure CI/CD -weight: 40 -keywords: php, CI/CD -description: Learn how to Configure CI/CD for your PHP application -aliases: - - /language/php/configure-ci-cd/ - - /guides/language/php/configure-ci-cd/ ---- - -## Prerequisites - -Complete all the previous sections of this guide, starting with [Containerize a PHP application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. - -## Overview - -In this section, you'll learn how to set up and use GitHub Actions to build and test your Docker image as well as push it to Docker Hub. You will complete the following steps: - -1. Create a new repository on GitHub. -2. Define the GitHub Actions workflow. -3. Run the workflow. - -## Step one: Create the repository - -Create a GitHub repository, configure the Docker Hub credentials, and push your source code. - -1. [Create a new repository](https://github.com/new) on GitHub. - -2. Open the repository **Settings**, and go to **Secrets and variables** > - **Actions**. - -3. Create a new **Repository variable** named `DOCKER_USERNAME` and your Docker ID as a value. - -4. Create a new [Personal Access Token (PAT)](/manuals/security/access-tokens.md#create-an-access-token) for Docker Hub. You can name this token `docker-tutorial`. Make sure access permissions include Read and Write. - -5. Add the PAT as a **Repository secret** in your GitHub repository, with the name - `DOCKERHUB_TOKEN`. - -6. In your local repository on your machine, run the following command to change - the origin to the repository you just created. Make sure you change - `your-username` to your GitHub username and `your-repository` to the name of - the repository you created. - - ```console - $ git remote set-url origin https://github.com/your-username/your-repository.git - ``` - -7. In your local repository on your machine, run the following command to rename - the branch to main. - - ```console - $ git branch -M main - ``` - -8. Run the following commands to stage, commit, and then push your local - repository to GitHub. - - ```console - $ git add -A - $ git commit -m "my first commit" - $ git push -u origin main - ``` - -## Step two: Set up the workflow - -Set up your GitHub Actions workflow for building, testing, and pushing the image -to Docker Hub. - -1. Go to your repository on GitHub and then select the **Actions** tab. - -2. Select **set up a workflow yourself**. - - This takes you to a page for creating a new GitHub actions workflow file in - your repository, under `.github/workflows/main.yml` by default. - -3. In the editor window, copy and paste the following YAML configuration. - - ```yaml - name: ci - - on: - push: - branches: - - main - - jobs: - build: - runs-on: ubuntu-latest - steps: - - name: Login to Docker Hub - uses: docker/login-action@{{% param "login_action_version" %}} - with: - username: ${{ vars.DOCKER_USERNAME }} - password: ${{ secrets.DOCKERHUB_TOKEN }} - - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} - - - name: Build and test - uses: docker/build-push-action@{{% param "build_push_action_version" %}} - with: - target: test - load: true - - - name: Build and push - uses: docker/build-push-action@{{% param "build_push_action_version" %}} - with: - platforms: linux/amd64,linux/arm64 - push: true - target: final - tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest - ``` - - For more information about the YAML syntax for `docker/build-push-action`, - refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). - -## Step three: Run the workflow - -Save the workflow file and run the job. - -1. Select **Commit changes...** and push the changes to the `main` branch. - - After pushing the commit, the workflow starts automatically. - -2. Go to the **Actions** tab. It displays the workflow. - - Selecting the workflow shows you the breakdown of all the steps. - -3. When the workflow is complete, go to your - [repositories on Docker Hub](https://hub.docker.com/repositories). - - If you see the new repository in that list, it means the GitHub Actions - successfully pushed the image to Docker Hub. - -## Summary - -In this section, you learned how to set up a GitHub Actions workflow for your application. - -Related information: - -- [Introduction to GitHub Actions](/guides/gha.md) -- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) -- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) - -## Next steps - -Next, learn how you can locally test and debug your workloads on Kubernetes before deploying. diff --git a/content/guides/php/containerize.md b/content/guides/php/containerize.md deleted file mode 100644 index 20f4987af1de..000000000000 --- a/content/guides/php/containerize.md +++ /dev/null @@ -1,283 +0,0 @@ ---- -title: Containerize a PHP application -linkTitle: Containerize your app -weight: 10 -keywords: php, containerize, initialize, apache, composer -description: Learn how to containerize a PHP application. -aliases: - - /language/php/containerize/ - - /guides/language/php/containerize/ ---- - -## Prerequisites - -- You have installed the latest version of [Docker - Desktop](/get-started/get-docker.md). -- You have a [git client](https://git-scm.com/downloads). The examples in this - section use a command-line based git client, but you can use any client. - -## Overview - -This section walks you through containerizing and running a PHP -application. - -## Get the sample applications - -In this guide, you will use a pre-built PHP application. The application uses Composer for library dependency management. You'll serve the application via an Apache web server. - -Open a terminal, change directory to a directory that you want to work in, and -run the following command to clone the repository. - -```console -$ git clone https://github.com/docker/docker-php-sample -``` - -The sample application is a basic hello world application and an application that increments a counter in a database. In addition, the application uses PHPUnit for testing. - -## Create Docker assets - -Now that you have an application, you can create the necessary Docker assets to -containerize it. - -> [!TIP] -> -> [Gordon](/ai/gordon/), Docker's AI assistant, can generate Docker assets for your project. Ask Gordon to create a Dockerfile, Compose file, and `.dockerignore` tailored to your application. - -Create the following files in your `docker-php-sample` directory. - -```dockerfile {collapse=true,title=Dockerfile} -# syntax=docker/dockerfile:1 - -# Comments are provided throughout this file to help you get started. -# If you need more help, visit the Dockerfile reference guide at -# https://docs.docker.com/go/dockerfile-reference/ - -################################################################################ - -# Create a stage for installing app dependencies defined in Composer. -FROM composer:lts as deps - -WORKDIR /app - -# If your composer.json file defines scripts that run during dependency installation and -# reference your application source files, uncomment the line below to copy all the files -# into this layer. -# COPY . . - -# Download dependencies as a separate step to take advantage of Docker's caching. -# Leverage a bind mounts to composer.json and composer.lock to avoid having to copy them -# into this layer. -# Leverage a cache mount to /tmp/cache so that subsequent builds don't have to re-download packages. -RUN --mount=type=bind,source=composer.json,target=composer.json \ - --mount=type=bind,source=composer.lock,target=composer.lock \ - --mount=type=cache,target=/tmp/cache \ - composer install --no-dev --no-interaction - -################################################################################ - -# Create a new stage for running the application that contains the minimal -# runtime dependencies for the application. This often uses a different base -# image from the install or build stage where the necessary files are copied -# from the install stage. -# -# The example below uses the PHP Apache image as the foundation for running the app. -# By specifying the "8.2-apache" tag, it will also use whatever happens to be the -# most recent version of that tag when you build your Dockerfile. -# If reproducibility is important, consider using a specific digest SHA, like -# php@sha256:99cede493dfd88720b610eb8077c8688d3cca50003d76d1d539b0efc8cca72b4. -FROM php:8.2-apache as final - -# Your PHP application may require additional PHP extensions to be installed -# manually. For detailed instructions for installing extensions can be found, see -# https://github.com/docker-library/docs/tree/master/php#how-to-install-more-php-extensions -# The following code blocks provide examples that you can edit and use. -# -# Add core PHP extensions, see -# https://github.com/docker-library/docs/tree/master/php#php-core-extensions -# This example adds the apt packages for the 'gd' extension's dependencies and then -# installs the 'gd' extension. For additional tips on running apt-get, see -# https://docs.docker.com/go/dockerfile-aptget-best-practices/ -# RUN apt-get update && apt-get install -y \ -# libfreetype-dev \ -# libjpeg62-turbo-dev \ -# libpng-dev \ -# && rm -rf /var/lib/apt/lists/* \ -# && docker-php-ext-configure gd --with-freetype --with-jpeg \ -# && docker-php-ext-install -j$(nproc) gd -# -# Add PECL extensions, see -# https://github.com/docker-library/docs/tree/master/php#pecl-extensions -# This example adds the 'redis' and 'xdebug' extensions. -# RUN pecl install redis-5.3.7 \ -# && pecl install xdebug-3.2.1 \ -# && docker-php-ext-enable redis xdebug - -# Use the default production configuration for PHP runtime arguments, see -# https://github.com/docker-library/docs/tree/master/php#configuration -RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini" - -# Copy the app dependencies from the previous install stage. -COPY --from=deps app/vendor/ /var/www/html/vendor -# Copy the app files from the app directory. -COPY ./src /var/www/html - -# Switch to a non-privileged user (defined in the base image) that the app will run under. -# See https://docs.docker.com/go/dockerfile-user-best-practices/ -USER www-data -``` - -```yaml {collapse=true,title=compose.yaml} -# Comments are provided throughout this file to help you get started. -# If you need more help, visit the Docker Compose reference guide at -# https://docs.docker.com/go/compose-spec-reference/ - -# Here the instructions define your application as a service called "server". -# This service is built from the Dockerfile in the current directory. -# You can add other services your application may depend on here, such as a -# database or a cache. For examples, see the Awesome Compose repository: -# https://github.com/docker/awesome-compose -services: - server: - build: - context: . - ports: - - 9000:80 - -# The commented out section below is an example of how to define a PostgreSQL -# database that your application can use. `depends_on` tells Docker Compose to -# start the database before your application. The `db-data` volume persists the -# database data between container restarts. The `db-password` secret is used -# to set the database password. You must create `db/password.txt` and add -# a password of your choosing to it before running `docker compose up`. -# depends_on: -# db: -# condition: service_healthy -# db: -# image: postgres -# restart: always -# user: postgres -# secrets: -# - db-password -# volumes: -# - db-data:/var/lib/postgresql/data -# environment: -# - POSTGRES_DB=example -# - POSTGRES_PASSWORD_FILE=/run/secrets/db-password -# expose: -# - 5432 -# healthcheck: -# test: [ "CMD", "pg_isready" ] -# interval: 10s -# timeout: 5s -# retries: 5 -# volumes: -# db-data: -# secrets: -# db-password: -# file: db/password.txt -``` - -```text {collapse=true,title=".dockerignore"} -# Include any files or directories that you don't want to be copied to your -# container here (e.g., local build artifacts, temporary files, etc.). -# -# For more help, visit the .dockerignore file reference guide at -# https://docs.docker.com/go/build-context-dockerignore/ - -**/.classpath -**/.dockerignore -**/.env -**/.git -**/.gitignore -**/.project -**/.settings -**/.toolstarget -**/.vs -**/.vscode -**/.next -**/.cache -**/*.*proj.user -**/*.dbmdl -**/*.jfm -**/charts -**/docker-compose* -**/compose.y*ml -!**/composer.json -!**/composer.lock -**/Dockerfile* -**/node_modules -**/npm-debug.log -**/obj -**/secrets.dev.yaml -**/values.dev.yaml -**/vendor -LICENSE -README.md -``` - -You should now have the following contents in your `docker-php-sample` -directory. - -```text -├── docker-php-sample/ -│ ├── .git/ -│ ├── src/ -│ ├── tests/ -│ ├── .dockerignore -│ ├── .gitignore -│ ├── compose.yaml -│ ├── composer.json -│ ├── composer.lock -│ ├── Dockerfile -│ └── README.md -``` - -To learn more about these files, see the following: - -- [Dockerfile](/reference/dockerfile.md) -- [.dockerignore](/reference/dockerfile.md#dockerignore-file) -- [compose.yaml](/reference/compose-file/_index.md) - -## Run the application - -Inside the `docker-php-sample` directory, run the following command in a -terminal. - -```console -$ docker compose up --build -``` - -Open a browser and view the application at [http://localhost:9000/hello.php](http://localhost:9000/hello.php). You should see a simple hello world application. - -In the terminal, press `ctrl`+`c` to stop the application. - -### Run the application in the background - -You can run the application detached from the terminal by adding the `-d` -option. Inside the `docker-php-sample` directory, run the following command -in a terminal. - -```console -$ docker compose up --build -d -``` - -Open a browser and view the application at [http://localhost:9000/hello.php](http://localhost:9000/hello.php). You should see a simple hello world application. - -In the terminal, run the following command to stop the application. - -```console -$ docker compose down -``` - -For more information about Compose commands, see the [Compose CLI -reference](/reference/cli/docker/compose/). - -## Summary - -In this section, you learned how you can containerize and run a simple PHP -application using Docker. - -## Next steps - -In the next section, you'll learn how you can develop your application using -Docker containers. diff --git a/content/guides/php/deploy.md b/content/guides/php/deploy.md deleted file mode 100644 index bd45d5ec0171..000000000000 --- a/content/guides/php/deploy.md +++ /dev/null @@ -1,146 +0,0 @@ ---- -title: Test your PHP deployment -linkTitle: Test your deployment -weight: 50 -keywords: deploy, php, local, development -description: Learn how to deploy your application -aliases: - - /language/php/deploy/ - - /guides/language/php/deploy/ ---- - -## Prerequisites - -- Complete all the previous sections of this guide, starting with [Containerize - a PHP application](containerize.md). -- [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker - Desktop. - -## Overview - -In this section, you'll learn how to use Docker Desktop to deploy your -application to a fully-featured Kubernetes environment on your development -machine. This allows you to test and debug your workloads on Kubernetes locally -before deploying. - -## Create a Kubernetes YAML file - -In your `docker-php-sample` directory, create a file named -`docker-php-kubernetes.yaml`. Open the file in an IDE or text editor and add -the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker -username and the name of the repository that you created in [Configure CI/CD for -your PHP application](configure-ci-cd.md). - -```yaml -apiVersion: apps/v1 -kind: Deployment -metadata: - name: docker-php-demo - namespace: default -spec: - replicas: 1 - selector: - matchLabels: - hello-php: web - template: - metadata: - labels: - hello-php: web - spec: - containers: - - name: hello-site - image: DOCKER_USERNAME/REPO_NAME - imagePullPolicy: Always ---- -apiVersion: v1 -kind: Service -metadata: - name: php-entrypoint - namespace: default -spec: - type: NodePort - selector: - hello-php: web - ports: - - port: 80 - targetPort: 80 - nodePort: 30001 -``` - -In this Kubernetes YAML file, there are two objects, separated by the `---`: - -- A Deployment, describing a scalable group of identical pods. In this case, - you'll get just one replica, or copy of your pod. That pod, which is - described under `template`, has just one container in it. The container is - created from the image built by GitHub Actions in [Configure CI/CD for your - PHP application](configure-ci-cd.md). -- A NodePort service, which will route traffic from port 30001 on your host to - port 80 inside the pods it routes to, allowing you to reach your app - from the network. - -To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). - -## Deploy and check your application - -1. In a terminal, navigate to the `docker-php-sample` directory - and deploy your application to Kubernetes. - - ```console - $ kubectl apply -f docker-php-kubernetes.yaml - ``` - - You should see output that looks like the following, indicating your Kubernetes objects were created successfully. - - ```text - deployment.apps/docker-php-demo created - service/php-entrypoint created - ``` - -2. Make sure everything worked by listing your deployments. - - ```console - $ kubectl get deployments - ``` - - Your deployment should be listed as follows: - - ```text - NAME READY UP-TO-DATE AVAILABLE AGE - docker-php-demo 1/1 1 1 6s - ``` - - This indicates all of the pods are up and running. Do the same check for your services. - - ```console - $ kubectl get services - ``` - - You should get output like the following. - - ```text - NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE - kubernetes ClusterIP 10.96.0.1 443/TCP 7d22h - php-entrypoint NodePort 10.111.101.229 80:30001/TCP 33s - ``` - - In addition to the default `kubernetes` service, you can see your `php-entrypoint` service. The `php-entrypoint` service is accepting traffic on port 30001/TCP. - -3. Open a browser and visit your app at - [http://localhost:30001/hello.php](http://localhost:30001/hello.php). You - should see your application. - -4. Run the following command to tear down your application. - - ```console - $ kubectl delete -f docker-php-kubernetes.yaml - ``` - -## Summary - -In this section, you learned how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. - -Related information: - -- [Kubernetes documentation](https://kubernetes.io/docs/home/) -- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) -- [Swarm mode overview](/manuals/engine/swarm/_index.md) diff --git a/content/guides/php/develop.md b/content/guides/php/develop.md deleted file mode 100644 index 17ca0cc3a6d6..000000000000 --- a/content/guides/php/develop.md +++ /dev/null @@ -1,458 +0,0 @@ ---- -title: Use containers for PHP development -linkTitle: Develop your app -weight: 20 -keywords: php, development -description: Learn how to develop your PHP application locally using containers. -aliases: - - /language/php/develop/ - - /guides/language/php/develop/ ---- - -## Prerequisites - -Complete [Containerize a PHP application](containerize.md). - -## Overview - -In this section, you'll learn how to set up a development environment for your containerized application. This includes: - -- Adding a local database and persisting data -- Adding phpMyAdmin to interact with the database -- Configuring Compose to automatically update your running Compose services as - you edit and save your code -- Creating a development container that contains the dev dependencies - -## Add a local database and persist data - -You can use containers to set up local services, like a database. -To do this for the sample application, you'll need to do the following: - -- Update the `Dockerfile` to install extensions to connect to the database -- Update the `compose.yaml` file to add a database service and volume to persist data - -### Update the Dockerfile to install extensions - -To install PHP extensions, you need to update the `Dockerfile`. Open your -Dockerfile in an IDE or text editor and then update the contents. The following -`Dockerfile` includes one new line that installs the `pdo` and `pdo_mysql` -extensions. All comments have been removed. - -```dockerfile {hl_lines=11} -# syntax=docker/dockerfile:1 - -FROM composer:lts as deps -WORKDIR /app -RUN --mount=type=bind,source=composer.json,target=composer.json \ - --mount=type=bind,source=composer.lock,target=composer.lock \ - --mount=type=cache,target=/tmp/cache \ - composer install --no-dev --no-interaction - -FROM php:8.2-apache as final -RUN docker-php-ext-install pdo pdo_mysql -RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini" -COPY --from=deps app/vendor/ /var/www/html/vendor -COPY ./src /var/www/html -USER www-data -``` - -For more details about installing PHP extensions, see the [Official Docker Image for PHP](https://hub.docker.com/_/php). - -### Update the compose.yaml file to add a db and persist data - -Open the `compose.yaml` file in an IDE or text editor. You'll notice it -already contains commented-out instructions for a PostgreSQL database and volume. For this application, you'll use MariaDB. For more details about MariaDB, see the [MariaDB Official Docker image](https://hub.docker.com/_/mariadb). - -Open the `src/database.php` file in an IDE or text editor. You'll notice that it reads environment variables in order to connect to the database. - -In the `compose.yaml` file, you'll need to update the following: - -1. Uncomment and update the database instructions for MariaDB. -2. Add a secret to the server service to pass in the database password. -3. Add the database connection environment variables to the server service. -4. Uncomment the volume instructions to persist data. - -The following is the updated `compose.yaml` file. All comments have been removed. - -```yaml -services: - server: - build: - context: . - ports: - - 9000:80 - depends_on: - db: - condition: service_healthy - secrets: - - db-password - environment: - - PASSWORD_FILE_PATH=/run/secrets/db-password - - DB_HOST=db - - DB_NAME=example - - DB_USER=root - db: - image: mariadb - restart: always - user: root - secrets: - - db-password - volumes: - - db-data:/var/lib/mysql - environment: - - MARIADB_ROOT_PASSWORD_FILE=/run/secrets/db-password - - MARIADB_DATABASE=example - expose: - - 3306 - healthcheck: - test: - [ - "CMD", - "/usr/local/bin/healthcheck.sh", - "--su-mysql", - "--connect", - "--innodb_initialized", - ] - interval: 10s - timeout: 5s - retries: 5 -volumes: - db-data: -secrets: - db-password: - file: db/password.txt -``` - -> [!NOTE] -> -> To learn more about the instructions in the Compose file, see [Compose file -> reference](/reference/compose-file/). - -Before you run the application using Compose, notice that this Compose file uses -`secrets` and specifies a `password.txt` file to hold the database's password. -You must create this file as it's not included in the source repository. - -In the `docker-php-sample` directory, create a new directory named `db` and -inside that directory create a file named `password.txt`. Open `password.txt` in an IDE or text editor and add the following password. The password must be on a single line, with no additional lines in the file. - -```text -example -``` - -Save and close the `password.txt` file. - -You should now have the following in your `docker-php-sample` directory. - -```text -├── docker-php-sample/ -│ ├── .git/ -│ ├── db/ -│ │ └── password.txt -│ ├── src/ -│ ├── tests/ -│ ├── .dockerignore -│ ├── .gitignore -│ ├── compose.yaml -│ ├── composer.json -│ ├── composer.lock -│ ├── Dockerfile -│ └── README.md -``` - -Run the following command to start your application. - -```console -$ docker compose up --build -``` - -Open a browser and view the application at [http://localhost:9000/database.php](http://localhost:9000/database.php). You should see a simple web application with text and a counter that increments every time you refresh. - -Press `ctrl+c` in the terminal to stop your application. - -## Verify that data persists in the database - -In the terminal, run `docker compose rm` to remove your containers and then run `docker compose up` to run your application again. - -```console -$ docker compose rm -$ docker compose up --build -``` - -Refresh [http://localhost:9000/database.php](http://localhost:9000/database.php) in your browser and verify that the previous count still exists. Without a volume, the database data wouldn't persist after you remove the container. - -Press `ctrl+c` in the terminal to stop your application. - -## Add phpMyAdmin to interact with the database - -You can easily add services to your application stack by updating the `compose.yaml` file. - -Update your `compose.yaml` to add a new service for phpMyAdmin. For more details, see the [phpMyAdmin Official Docker Image](https://hub.docker.com/_/phpmyadmin). The following is the updated `compose.yaml` file. - -```yaml {hl_lines="42-49"} -services: - server: - build: - context: . - ports: - - 9000:80 - depends_on: - db: - condition: service_healthy - secrets: - - db-password - environment: - - PASSWORD_FILE_PATH=/run/secrets/db-password - - DB_HOST=db - - DB_NAME=example - - DB_USER=root - db: - image: mariadb - restart: always - user: root - secrets: - - db-password - volumes: - - db-data:/var/lib/mysql - environment: - - MARIADB_ROOT_PASSWORD_FILE=/run/secrets/db-password - - MARIADB_DATABASE=example - expose: - - 3306 - healthcheck: - test: - [ - "CMD", - "/usr/local/bin/healthcheck.sh", - "--su-mysql", - "--connect", - "--innodb_initialized", - ] - interval: 10s - timeout: 5s - retries: 5 - phpmyadmin: - image: phpmyadmin - ports: - - 8080:80 - depends_on: - - db - environment: - - PMA_HOST=db -volumes: - db-data: -secrets: - db-password: - file: db/password.txt -``` - -In the terminal, run `docker compose up` to run your application again. - -```console -$ docker compose up --build -``` - -Open [http://localhost:8080](http://localhost:8080) in your browser to access phpMyAdmin. Log in using `root` as the username and `example` as the password. You can now interact with the database through phpMyAdmin. - -Press `ctrl+c` in the terminal to stop your application. - -## Automatically update services - -Use Compose Watch to automatically update your running Compose services as you edit and save your code. For more details about Compose Watch, see [Use Compose Watch](/manuals/compose/how-tos/file-watch.md). - -Open your `compose.yaml` file in an IDE or text editor and then add the Compose Watch instructions. The following is the updated `compose.yaml` file. - -```yaml {hl_lines="17-21"} -services: - server: - build: - context: . - ports: - - 9000:80 - depends_on: - db: - condition: service_healthy - secrets: - - db-password - environment: - - PASSWORD_FILE_PATH=/run/secrets/db-password - - DB_HOST=db - - DB_NAME=example - - DB_USER=root - develop: - watch: - - action: sync - path: ./src - target: /var/www/html - db: - image: mariadb - restart: always - user: root - secrets: - - db-password - volumes: - - db-data:/var/lib/mysql - environment: - - MARIADB_ROOT_PASSWORD_FILE=/run/secrets/db-password - - MARIADB_DATABASE=example - expose: - - 3306 - healthcheck: - test: - [ - "CMD", - "/usr/local/bin/healthcheck.sh", - "--su-mysql", - "--connect", - "--innodb_initialized", - ] - interval: 10s - timeout: 5s - retries: 5 - phpmyadmin: - image: phpmyadmin - ports: - - 8080:80 - depends_on: - - db - environment: - - PMA_HOST=db -volumes: - db-data: -secrets: - db-password: - file: db/password.txt -``` - -Run the following command to run your application with Compose Watch. - -```console -$ docker compose watch -``` - -Open a browser and verify that the application is running at [http://localhost:9000/hello.php](http://localhost:9000/hello.php). - -Any changes to the application's source files on your local machine will now be -immediately reflected in the running container. - -Open `hello.php` in an IDE or text editor and update the string `Hello, world!` to `Hello, Docker!`. - -Save the changes to `hello.php` and then wait a few seconds for the application to sync. Refresh [http://localhost:9000/hello.php](http://localhost:9000/hello.php) in your browser and verify that the updated text appears. - -Press `ctrl+c` in the terminal to stop Compose Watch. Run `docker compose down` in the terminal to stop the application. - -## Create a development container - -At this point, when you run your containerized application, Composer isn't installing the dev dependencies. While this small image is good for production, it lacks the tools and dependencies you may need when developing and it doesn't include the `tests` directory. You can use multi-stage builds to build stages for both development and production in the same Dockerfile. For more details, see [Multi-stage builds](/manuals/build/building/multi-stage.md). - -In the `Dockerfile`, you'll need to update the following: - -1. Split the `deps` staged into two stages. One stage for production - (`prod-deps`) and one stage (`dev-deps`) to install development dependencies. -2. Create a common `base` stage. -3. Create a new `development` stage for development. -4. Update the `final` stage to copy dependencies from the new `prod-deps` stage. - -The following is the `Dockerfile` before and after the changes. - -{{< tabs >}} -{{< tab name="Before" >}} - -```dockerfile -# syntax=docker/dockerfile:1 - -FROM composer:lts as deps -WORKDIR /app -RUN --mount=type=bind,source=composer.json,target=composer.json \ - --mount=type=bind,source=composer.lock,target=composer.lock \ - --mount=type=cache,target=/tmp/cache \ - composer install --no-dev --no-interaction - -FROM php:8.2-apache as final -RUN docker-php-ext-install pdo pdo_mysql -RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini" -COPY --from=deps app/vendor/ /var/www/html/vendor -COPY ./src /var/www/html -USER www-data -``` - -{{< /tab >}} -{{< tab name="After" >}} - -```dockerfile -# syntax=docker/dockerfile:1 - -FROM composer:lts as prod-deps -WORKDIR /app -RUN --mount=type=bind,source=./composer.json,target=composer.json \ - --mount=type=bind,source=./composer.lock,target=composer.lock \ - --mount=type=cache,target=/tmp/cache \ - composer install --no-dev --no-interaction - -FROM composer:lts as dev-deps -WORKDIR /app -RUN --mount=type=bind,source=./composer.json,target=composer.json \ - --mount=type=bind,source=./composer.lock,target=composer.lock \ - --mount=type=cache,target=/tmp/cache \ - composer install --no-interaction - -FROM php:8.2-apache as base -RUN docker-php-ext-install pdo pdo_mysql -COPY ./src /var/www/html - -FROM base as development -COPY ./tests /var/www/html/tests -RUN mv "$PHP_INI_DIR/php.ini-development" "$PHP_INI_DIR/php.ini" -COPY --from=dev-deps app/vendor/ /var/www/html/vendor - -FROM base as final -RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini" -COPY --from=prod-deps app/vendor/ /var/www/html/vendor -USER www-data -``` - -{{< /tab >}} -{{< /tabs >}} - -Update your `compose.yaml` file by adding an instruction to target the -development stage. - -The following is the updated section of the `compose.yaml` file. - -```yaml {hl_lines=5} -services: - server: - build: - context: . - target: development - # ... -``` - -Your containerized application will now install the dev dependencies. - -Run the following command to start your application. - -```console -$ docker compose up --build -``` - -Open a browser and view the application at [http://localhost:9000/hello.php](http://localhost:9000/hello.php). You should still see the simple "Hello, Docker!" application. - -Press `ctrl+c` in the terminal to stop your application. - -While the application appears the same, you can now make use of the dev dependencies. Continue to the next section to learn how you can run tests using Docker. - -## Summary - -In this section, you took a look at setting up your Compose file to add a local -database and persist data. You also learned how to use Compose Watch to automatically sync your application when you update your code. And finally, you learned how to create a development container that contains the dependencies needed for development. - -Related information: - -- [Compose file reference](/reference/compose-file/) -- [Compose file watch](/manuals/compose/how-tos/file-watch.md) -- [Dockerfile reference](/reference/dockerfile.md) -- [Official Docker Image for PHP](https://hub.docker.com/_/php) - -## Next steps - -In the next section, you'll learn how to run unit tests using Docker. diff --git a/content/guides/php/run-tests.md b/content/guides/php/run-tests.md deleted file mode 100644 index 7ddad4d72fb8..000000000000 --- a/content/guides/php/run-tests.md +++ /dev/null @@ -1,118 +0,0 @@ ---- -title: Run PHP tests in a container -linkTitle: Run your tests -weight: 30 -keywords: php, test -description: Learn how to run your PHP tests in a container. -aliases: - - /language/php/run-tests/ - - /guides/language/php/run-tests/ ---- - -## Prerequisites - -Complete all the previous sections of this guide, starting with [Containerize a PHP application](containerize.md). - -## Overview - -Testing is an essential part of modern software development. Testing can mean a -lot of things to different development teams. There are unit tests, integration -tests and end-to-end testing. In this guide you take a look at running your unit -tests in Docker when developing and when building. - -## Run tests when developing locally - -The sample application already has a PHPUnit test inside the `tests` directory. When developing locally, you can use Compose to run your tests. - -Run the following command in the `docker-php-sample` directory to run the tests inside a container. - -```console -$ docker compose run --build --rm server ./vendor/bin/phpunit tests/HelloWorldTest.php -``` - -You should see output that contains the following. - -```console -Hello, Docker!PHPUnit 9.6.13 by Sebastian Bergmann and contributors. - -. 1 / 1 (100%) - -Time: 00:00.003, Memory: 4.00 MB - -OK (1 test, 1 assertion) -``` - -To learn more about the command, see [docker compose run](/reference/cli/docker/compose/run/). - -## Run tests when building - -To run your tests when building, you need to update your Dockerfile. Create a new test stage that runs the tests. - -The following is the updated Dockerfile. - -```dockerfile {hl_lines="26-28"} -# syntax=docker/dockerfile:1 - -FROM composer:lts as prod-deps -WORKDIR /app -RUN --mount=type=bind,source=./composer.json,target=composer.json \ - --mount=type=bind,source=./composer.lock,target=composer.lock \ - --mount=type=cache,target=/tmp/cache \ - composer install --no-dev --no-interaction - -FROM composer:lts as dev-deps -WORKDIR /app -RUN --mount=type=bind,source=./composer.json,target=composer.json \ - --mount=type=bind,source=./composer.lock,target=composer.lock \ - --mount=type=cache,target=/tmp/cache \ - composer install --no-interaction - -FROM php:8.2-apache as base -RUN docker-php-ext-install pdo pdo_mysql -COPY ./src /var/www/html - -FROM base as development -COPY ./tests /var/www/html/tests -RUN mv "$PHP_INI_DIR/php.ini-development" "$PHP_INI_DIR/php.ini" -COPY --from=dev-deps app/vendor/ /var/www/html/vendor - -FROM development as test -WORKDIR /var/www/html -RUN ./vendor/bin/phpunit tests/HelloWorldTest.php - -FROM base as final -RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini" -COPY --from=prod-deps app/vendor/ /var/www/html/vendor -USER www-data -``` - -Run the following command to build an image using the test stage as the target and view the test results. Include `--progress plain` to view the build output, `--no-cache` to ensure the tests always run, and `--target test` to target the test stage. - -```console -$ docker build -t php-docker-image-test --progress plain --no-cache --target test . -``` - -You should see output containing the following. - -```console -#18 [test 2/2] RUN ./vendor/bin/phpunit tests/HelloWorldTest.php -#18 0.385 Hello, Docker!PHPUnit 9.6.13 by Sebastian Bergmann and contributors. -#18 0.392 -#18 0.394 . 1 / 1 (100%) -#18 0.395 -#18 0.395 Time: 00:00.003, Memory: 4.00 MB -#18 0.395 -#18 0.395 OK (1 test, 1 assertion) -``` - -## Summary - -In this section, you learned how to run tests when developing locally using Compose and how to run tests when building your image. - -Related information: - -- [docker compose run](/reference/cli/docker/compose/run/) - -## Next steps - -Next, you’ll learn how to set up a CI/CD pipeline using GitHub Actions. diff --git a/content/guides/postgresql/_index.md b/content/guides/postgresql/_index.md index 36d32a728371..869e976739af 100644 --- a/content/guides/postgresql/_index.md +++ b/content/guides/postgresql/_index.md @@ -6,10 +6,1166 @@ keywords: Docker, getting started, postgresql, language summary: | This guide explains how to containerize PostgreSQL databases using Docker. -toc_min: 1 -toc_max: 2 -tags: [databases] +aliases: + - /guides/postgresql/advanced-configuration-and-initialization/ + - /guides/postgresql/companions-for-postgresql/ + - /guides/postgresql/immediate-setup-and-data-persistence/ + - /guides/postgresql/networking-and-connectivity/ params: + tags: [databases] time: 20 minutes --- + +## Immediate setup & data persistence + +This guide gets you from zero to a running PostgreSQL container in under five minutes, then explains how to keep your data safe across container restarts and removals. + +### Overview + +Running PostgreSQL in Docker requires understanding one critical concept: containers are ephemeral, but your data shouldn't be. This guide covers: + +- Starting PostgreSQL with a single command +- Understanding why containers lose data by default +- Configuring volumes for persistent storage +- Translating your setup to Docker Compose + +### Quick start (minimal viable container) + +> [!NOTE] +> +> [Docker Hardened Images (DHIs)](https://docs.docker.com/dhi/) are minimal, secure, and production-ready container base and application images maintained by Docker. DHIs are recommended whenever it is possible for better security. They are designed to reduce vulnerabilities and simplify compliance, freely available to everyone with no subscription required, no usage restrictions, and no vendor lock-in. + +Run PostgreSQL immediately with this single command: + +{{< tabs >}} +{{< tab name="Using DHIs" >}} + +You must authenticate to dhi.io before you can pull Docker Hardened Images. Run `docker login dhi.io` to authenticate. + +```console +docker run --rm --name postgres-dev \ + -e POSTGRES_PASSWORD=mysecretpassword \ + -p 5432:5432 \ + -d dhi.io/postgres:18 +``` + +{{< /tab >}} + +{{< tab name="Using DOIs" >}} + +```console +$ docker run --rm --name postgres-dev \ + -e POSTGRES_PASSWORD=mysecretpassword \ + -p 5432:5432 \ + -d postgres:18 +``` + +{{< /tab >}} +{{< /tabs >}} + +#### Understanding the flags + +| Flag | Purpose | +|------|---------| +| `--rm` | Automatically removes the container when it stops | +| `--name postgres-dev` | Assigns a memorable name instead of a random string | +| `-e POSTGRES_PASSWORD=...` | Sets the superuser password (required) | +| `-p 5432:5432` | Maps host port 5432 to container port 5432 | +| `-d` | Runs the container in the background (detached mode) | + +Verify the container is running: + +```console +$ docker ps --filter name=postgres-dev +CONTAINER ID IMAGE COMMAND STATUS PORTS NAMES +a1b2c3d4e5f6 postgres:18 "docker-entrypoint.s…" Up 2 seconds 0.0.0.0:5432->5432/tcp postgres-dev +``` + +Connect using `psql` from inside the container: + +```console +$ docker exec -it postgres-dev psql -U postgres +psql (18.0) +Type "help" for help. + +postgres=# +``` + +You now have a working PostgreSQL instance. But there's a problem—stop this container and your data disappears. + +### The data persistence problem + +Containers use an ephemeral filesystem. When a container is removed, everything inside it, including your database files, is deleted. + +Demonstrate this yourself: + +{{< tabs >}} +{{< tab name="Using DHIs" >}} + +```console +$ docker exec postgres-dev psql -U postgres -c "CREATE DATABASE testdb;" +CREATE DATABASE + +$ docker exec postgres-dev psql -U postgres -c "\l" | grep testdb + testdb | postgres | UTF8 | libc | en_US.utf8 | en_US.utf8 | | | + +$ docker stop postgres-dev +postgres-dev + +$ docker run --rm --name postgres-dev \ + -e POSTGRES_PASSWORD=mysecretpassword \ + -p 5432:5432 \ + -d dhi.io/postgres:18 + +$ docker exec postgres-dev psql -U postgres -c "\l" | grep testdb +(no output - database is gone) +``` + +{{< /tab >}} + +{{< tab name="Using DOIs" >}} + +```console +$ docker exec postgres-dev psql -U postgres -c "CREATE DATABASE testdb;" +CREATE DATABASE + +$ docker exec postgres-dev psql -U postgres -c "\l" | grep testdb + testdb | postgres | UTF8 | libc | en_US.utf8 | en_US.utf8 | | | + +$ docker stop postgres-dev +postgres-dev + +$ docker run --rm --name postgres-dev \ + -e POSTGRES_PASSWORD=mysecretpassword \ + -p 5432:5432 \ + -d postgres:18 + +$ docker exec postgres-dev psql -U postgres -c "\l" | grep testdb +(no output - database is gone) +``` + +{{< /tab >}} +{{< /tabs >}} + +Your `testdb` database vanished because the new container started with a fresh filesystem. This is expected behavior—and exactly why volumes exist. + +### Named volumes + +Named volumes are Docker-managed storage locations that persist independently of containers. Docker handles the filesystem location, permissions, and lifecycle. + +Create a container with a named volume: + +{{< tabs >}} +{{< tab name="Using DHIs" >}} + +You must authenticate to dhi.io before you can pull Docker Hardened Images. Run `docker login dhi.io` to authenticate. + +```console +$ docker run --rm --name postgres-dev \ + -e POSTGRES_PASSWORD=mysecretpassword \ + -p 5432:5432 \ + -v postgres_data:/var/lib/postgresql \ + -d dhi.io/postgres:18 +``` + +{{< /tab >}} + +{{< tab name="Using DOIs" >}} + +```console +$ docker run --rm --name postgres-dev \ + -e POSTGRES_PASSWORD=mysecretpassword \ + -p 5432:5432 \ + -v postgres_data:/var/lib/postgresql \ + -d postgres:18 +``` + +{{< /tab >}} +{{< /tabs >}} + + +The `-v postgres_data:/var/lib/postgresql` flag mounts a named volume called `postgres_data` to PostgreSQL's data directory. If the volume doesn't exist, Docker creates it automatically. + +> [!NOTE] +> +> PostgreSQL 18+ stores data in a version-specific subdirectory under `/var/lib/postgresql`. Mounting at this level (rather than `/var/lib/postgresql/data`) allows for easier upgrades using `pg_upgrade --link`. + +#### Verify persistence works + +To verify data persistence, repeat the previous test, but this time with the named volume attached in place. + +{{< tabs >}} +{{< tab name="Using DHIs" >}} + +```console +$ docker exec postgres-dev psql -U postgres -c "CREATE DATABASE testdb;" +CREATE DATABASE + +$ docker stop postgres-dev +postgres-dev + +$ docker run --rm --name postgres-dev \ + -e POSTGRES_PASSWORD=mysecretpassword \ + -p 5432:5432 \ + -v postgres_data:/var/lib/postgresql \ + -d dhi.io/postgres:18 + +$ docker exec postgres-dev psql -U postgres -c "\l" | grep testdb + testdb | postgres | UTF8 | libc | en_US.utf8 | en_US.utf8 | | | +``` + +{{< /tab >}} + +{{< tab name="Using DOIs" >}} + +```console +$ docker exec postgres-dev psql -U postgres -c "CREATE DATABASE testdb;" +CREATE DATABASE + +$ docker stop postgres-dev +postgres-dev + +$ docker run --rm --name postgres-dev \ + -e POSTGRES_PASSWORD=mysecretpassword \ + -p 5432:5432 \ + -v postgres_data:/var/lib/postgresql \ + -d postgres:18 + +$ docker exec postgres-dev psql -U postgres -c "\l" | grep testdb + testdb | postgres | UTF8 | libc | en_US.utf8 | en_US.utf8 | | | +``` + +{{< /tab >}} + +{{< /tabs >}} + +If you see `testdb` in the output, persistence works: The database survived because the volume preserved the data directory. + +#### Managing volumes + +List all volumes: + +```console +$ docker volume ls --filter name=postgres_data +DRIVER VOLUME NAME +local postgres_data +``` + +Inspect a volume to see its details: + +```console +$ docker volume inspect postgres_data +[ + { + "CreatedAt": "2025-01-05T10:30:00Z", + "Driver": "local", + "Labels": null, + "Mountpoint": "/var/lib/docker/volumes/postgres_data/_data", + "Name": "postgres_data", + "Options": null, + "Scope": "local" + } +] +``` + +Remove an unused volume (warning: this deletes all data): + +```console +$ docker volume rm postgres_data +``` + +### Bind mounts (alternative) + +Bind mounts map a specific host directory to a container path. Unlike named volumes, you control exactly where data lives on the host filesystem. + +Create a directory on your host machine to store Postgres data. + +{{< tabs >}} +{{< tab name="Using DHIs" >}} + +```console +mkdir -p ~/postgres-data && sudo chown -R 999:999 ~/postgres-data +``` + +Run Postgres using a bind mount. + +```console +docker run --rm --name postgres-dev \ + -e POSTGRES_PASSWORD=mysecretpassword \ + -p 5432:5432 \ + -v ~/postgres-data:/var/lib/postgresql \ + -d dhi.io/postgres:18 +``` + +{{< /tab >}} + +{{< tab name="Using DOIs" >}} + +```console +$ mkdir -p ~/postgres-data +``` + +Run Postgres using a bind mount. + +```console +$ docker run --rm --name postgres-dev \ + -e POSTGRES_PASSWORD=mysecretpassword \ + -p 5432:5432 \ + -v ~/postgres-data:/var/lib/postgresql \ + -d postgres:18 +``` + +{{< /tab >}} +{{< /tabs >}} + +#### When to use bind mounts + +Bind mounts are useful when you need direct filesystem access to the data directory for backup scripts that read files directly, when integrating with host-level monitoring tools, or when specific permission requirements exist. For most development and production scenarios, named volumes are simpler and less error-prone. + +#### Common bind mount issues + +Permission errors are the most frequent problem with bind mounts. PostgreSQL runs as user `postgres` (UID 999) inside the container. If your host directory has restrictive permissions, the container fails to start. + +Check logs if the container exits immediately: + +```console +$ docker logs postgres-dev +``` + +### Docker Compose configuration + +Docker Compose captures your entire configuration in a file, making setups reproducible and easier to manage as complexity grows. + +Create a `compose.yaml` file: + +```yaml +services: + db: + image: postgres:18 + container_name: postgres-dev + environment: + POSTGRES_PASSWORD: mysecretpassword + ports: + - "5432:5432" + volumes: + - postgres_data:/var/lib/postgresql + +volumes: + postgres_data: +``` + +Start the database: + +```console +$ docker compose up -d +``` + +Stop and remove containers (volume persists): + +```console +$ docker compose down +``` + +Alternatively, you can stop, remove containers, and delete the volume: + +```console +$ docker compose down -v +``` + +This compose file becomes the foundation for adding initialization scripts, performance tuning, and companion services covered in subsequent guides. + +#### Environment variables reference + +The official PostgreSQL image supports these environment variables: + +| Variable | Required | Description | +|----------|----------|-------------| +| `POSTGRES_PASSWORD` | Yes | Superuser password | +| `POSTGRES_USER` | No | Superuser name (default: `postgres`) | +| `POSTGRES_DB` | No | Default database name (default: value of `POSTGRES_USER`) | + +### Next steps + +With persistent storage configured, you're ready to customize PostgreSQL further. The next chapter of the guide covers: + +- Automated schema creation with initialization scripts +- Performance tuning for containerized workloads +- Timezone and locale configuration + +## Advanced Configuration and Initialization + +With persistent storage configured in the previous section, you're ready to customize PostgreSQL for real-world use. This guide covers advanced configuration techniques for running PostgreSQL in Docker containers, including automated database initialization, performance tuning, and timezone configuration. + +### Overview + +While PostgreSQL containers can be started quickly with default settings, production environments require customized configurations. This guide explains how to: + +- Automate database, schema, and user creation during container startup +- Tune PostgreSQL performance parameters for containerized workloads +- Configure timezone and locale settings + +### Initialization scripts + +The official PostgreSQL Docker image supports running initialization scripts automatically when the container starts for the first time. Any files placed in the `/docker-entrypoint-initdb.d/` directory are executed in alphabetical order. + +#### How initialization works + +When the container starts, it checks whether the PostgreSQL data directory is empty. If the directory already contains data, PostgreSQL starts immediately without running any initialization. If the directory is empty, the container runs `initdb` to create a new database cluster, then executes all scripts in `/docker-entrypoint-initdb.d/` in alphabetical order before starting PostgreSQL. + +#### Supported file formats + +| Format | Description | +|--------|-------------| +| `.sql` | SQL commands executed directly | +| `.sql.gz` | Gzip-compressed SQL files | +| `.sh` | Shell scripts executed with bash | + +> [!IMPORTANT] +> +> Initialization scripts only run when the PostgreSQL data directory (`/var/lib/postgresql/data`) is empty. If you mount a volume containing existing data, initialization is skipped. This behavior prevents overwriting existing databases. + +### Mounting initialization scripts + +Use Docker Compose to mount initialization scripts into the container. First, create a project directory: + +```console +$ mkdir -p postgres-project/init-db +$ cd postgres-project +``` + +Create a `compose.yaml` file: + +```yaml +services: + db: + image: postgres:18 + volumes: + - ./init-db:/docker-entrypoint-initdb.d + - postgres_data:/var/lib/postgresql + environment: + POSTGRES_PASSWORD: mysecretpassword + +volumes: + postgres_data: +``` + +All scripts in the `./init-db` directory execute when the container starts for the first time. This is great for bootstrapping databases. + +### Initialization script example + +Create a file named `init.sql` in your `init-db` directory: + +```sql +CREATE TABLE users ( + id SERIAL PRIMARY KEY, + email VARCHAR(255) UNIQUE NOT NULL, + name VARCHAR(100) NOT NULL, + created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP +); +``` + +This script runs automatically when the container starts for the first time, creating your initial database schema. + +> [!NOTE] +> +> Ensure initialization scripts have proper read permissions. If you encounter "Permission denied" errors, run `chmod 644 init-db/*.sql` to make the files readable by the container. + +### Performance tuning + +Default PostgreSQL settings are conservative to work on systems with limited resources. For production workloads, you should tune these parameters based on your container's allocated resources. + +#### Method 1: Custom configuration file + +For complete control, mount a custom `postgresql.conf` file. First, extract the default configuration: + +```console +$ docker run -i --rm postgres:18 cat /usr/share/postgresql/postgresql.conf.sample > my-postgres.conf +``` + +Edit `my-postgres.conf` with your desired settings, then mount it in your Compose file: + +```yaml +services: + db: + image: postgres:18 + volumes: + - ./my-postgres.conf:/etc/postgresql/postgresql.conf + - ./init-db:/docker-entrypoint-initdb.d + - postgres_data:/var/lib/postgresql + command: postgres -c config_file=/etc/postgresql/postgresql.conf + environment: + POSTGRES_PASSWORD: mysecretpassword + +volumes: + postgres_data: +``` + +### Key configuration parameters + +The following tables list important `postgresql.conf` parameters for containerized PostgreSQL deployments. + +#### Connection settings + +| Parameter | Description | Default | +|-----------|-------------|---------| +| `listen_addresses` | IP addresses to listen on | `localhost` | +| `port` | TCP port number | `5432` | +| `max_connections` | Maximum concurrent connections | `100` | + +#### Memory settings + +| Parameter | Description | Recommended starting value | +|-----------|-------------|---------------------------| +| `shared_buffers` | Shared memory for caching | 25% of container memory | +| `work_mem` | Memory per query operation | 4MB - 64MB | +| `maintenance_work_mem` | Memory for VACUUM, CREATE INDEX | 64MB - 256MB | +| `effective_cache_size` | Planner's cache size estimate | 50-75% of container memory | + +##### Docker memory limits + +When tuning memory parameters, set explicit memory limits on your container using `deploy.resources.limits.memory` in Compose or `--memory` with `docker run`. Without limits, PostgreSQL sees the host's total RAM and may allocate more than intended. For example, if your container should use 4GB maximum, set `shared_buffers` to approximately 1GB (25%). + +#### I/O settings + +| Parameter | Description | Recommended starting value | +|-----------|-------------|---------------------------| +| `effective_io_concurrency` | Concurrent disk I/O operations | `200` for SSDs, `2` for HDDs | + +#### Timeout settings + +| Parameter | Description | Default | +|-----------|-------------|---------| +| `statement_timeout` | Max time for any statement | `0` (disabled) | +| `lock_timeout` | Max time to wait for a lock | `0` (disabled) | +| `deadlock_timeout` | Time before checking for deadlock | `1s` | +| `transaction_timeout` | Max time for a transaction | `0` (disabled) | + +> [!NOTE] +> +> Setting `shared_buffers` too high in a container can exceed kernel shared memory limits. Use no more than 25-30% of the container's memory limit. + +### Timezone and locale configuration + +Proper localization ensures timestamps and sorting behave correctly for your application's users. + +```yaml +services: + db: + image: postgres:18 + volumes: + - postgres_data:/var/lib/postgresql + - /etc/localtime:/etc/localtime:ro + - /etc/timezone:/etc/timezone:ro + environment: + POSTGRES_PASSWORD: mysecretpassword + TZ: America/New_York + +volumes: + postgres_data: +``` + +Alternatively, set the timezone using a PostgreSQL command-line parameter: + +```yaml +services: + db: + image: postgres:18 + command: ["postgres", "-c", "timezone=America/New_York"] + environment: + POSTGRES_PASSWORD: mysecretpassword +``` + +#### Setting the locale + +Specify locale settings during database initialization using the `POSTGRES_INITDB_ARGS` environment variable: + +```yaml +services: + db: + image: postgres:18 + volumes: + - postgres_data:/var/lib/postgresql + environment: + POSTGRES_PASSWORD: mysecretpassword + POSTGRES_INITDB_ARGS: "--encoding=UTF8 --lc-collate=en_US.UTF-8 --lc-ctype=en_US.UTF-8" + +volumes: + postgres_data: +``` + +This affects collation (sorting) and character processing behavior. Changing this variable after database creation has no effect—it only applies during the first run when the data directory is initialized. + +### Connecting to the database + +You can interact with PostgreSQL running in a container even without `psql` installed on your host machine. + +#### Interactive shell + +Open a `psql` session inside the container: + +```console +$ docker exec -it postgres-container psql -U postgres +``` + +Connect to a specific database: + +```console +$ docker exec -it postgres-container psql -U postgres -d mydb +``` + +## Networking and connectivity + +This guide covers two common ways to connect to PostgreSQL running in Docker: + +- Container-to-container: Connect from your application container to PostgreSQL over a private Docker network. No ports need to be exposed to the host. +- Host-to-container: Connect from your laptop or development machine using `localhost` and a published port. + +Prerequisite: This guide assumes you have PostgreSQL running with persistent storage. If you don't, follow the [Immediate Setup & Data Persistence](/guides/postgresql/immediate-setup-and-data-persistence/) guide first. + +### Internal network access (container-to-container) + +When your application runs in another container, connecting to PostgreSQL through a user-defined bridge network is the recommended approach. This setup provides automatic DNS resolution, so your application can connect to PostgreSQL using the container name as the hostname, without needing to track IP addresses. + +> [!NOTE] +> Why not use the default bridge network? While containers on the default bridge network can communicate, they can only do so by IP address. Since container IP addresses change when containers restart, this would require updating your PostgreSQL connection strings each time. User-defined bridge networks solve this by providing automatic DNS resolution, ensuring your PostgreSQL connection strings remain stable even if containers restart and receive new IP addresses. + +Here's a quick comparison: + +> [!NOTE] +> +> The following examples show the difference in approach. To actually test this, follow the steps in this guide to set up containers on the appropriate networks first. + +With the default bridge network, you'd need to find the IP address first: +```bash +# Get the container's IP address (changes on restart) +docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' postgres-dev +# Output: 172.17.0.2 + +# Then connect using that IP address from another container +# (No --network flag needed - containers default to bridge network) +docker run --rm -it \ + -e PGPASSWORD=mysecretpassword \ + postgres:18 \ + psql -h 172.17.0.2 -U postgres +``` + +With a user-defined network, you simply use the container name: +```bash +# Container name works directly - no IP lookup needed +docker run --rm -it \ + --network my-app-net \ + -e PGPASSWORD=mysecretpassword \ + postgres:18 \ + psql -h postgres-dev -U postgres +``` + +#### Step 1: Create a user-defined network + +```bash +docker network create my-app-net + +# Example Output +ab7f984be43a0ca15534a9ee568716ddbe869a5875077fad3ef3192e3af7d288 + +docker network ls +# Output +ab7f984be43a my-app-net bridge local + + +``` + +#### Step 2: Run PostgreSQL on that network (no port publishing) + +Notice there is no `-p 5432:5432` here. This keeps PostgreSQL internal to Docker and not accessible from the host machine, which is more secure for production environments. + +```bash +docker run -d --name postgres-dev \ + --network my-app-net \ + -e POSTGRES_PASSWORD=mysecretpassword \ + -v postgres_data:/var/lib/postgresql \ + postgres:18 + + # Output +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +6d351ed89efc postgres:18 "docker-entrypoint.s…" 9 seconds ago Up 8 seconds 5432/tcp postgres-dev + +``` + +#### Step 3: Connect from another container using the Postgres container name + +You can test connectivity with a temporary `psql` client container: + +```bash +docker run --rm -it \ + --network my-app-net \ + -e PGPASSWORD=mysecretpassword \ + postgres:18 \ + psql -h postgres-dev -U postgres +``` + +Key point: `-h postgres-dev` works because Docker DNS resolves the container name on a user-defined network. The container name acts as the hostname. + +#### Connection string examples + +When connecting from your application container, use these PostgreSQL connection strings: + +- PostgreSQL URI format: + This is the standard PostgreSQL connection URI format that combines all connection parameters into a single string, widely supported by PostgreSQL clients and libraries. + + ```bash + postgresql://postgres:mysecretpassword@postgres-dev:5432/postgres + ``` + + This command demonstrates passing a PostgreSQL URI connection string as an environment variable to a container, which your application can then read to connect to the database. + + Example usage in a Docker run command: + ```bash + docker run --rm -it \ + --network my-app-net \ + -e DATABASE_URL="postgresql://postgres:mysecretpassword@postgres-dev:5432/postgres" \ + alpine:latest \ + sh -c 'echo "DATABASE_URL is set to: $DATABASE_URL"' + ``` + + +- PostgreSQL connection parameters: + This format uses key-value pairs separated by spaces, which many PostgreSQL client libraries accept as an alternative to URI format. + ```bash + host=postgres-dev + port=5432 + user=postgres + password=mysecretpassword + dbname=postgres + ``` + + Example usage in application code (Python with psycopg2): + ```python + conn = psycopg2.connect( + host="postgres-dev", + port=5432, + user="postgres", + password="mysecretpassword", + dbname="postgres" + ) + ``` + +- Connecting to a specific database: + Replace the database name in the connection string to connect to a specific database instead of the default `postgres` database. + If you created a custom database (e.g., `testdb`), use: + ```bash + postgresql://postgres:mysecretpassword@postgres-dev:5432/testdb + ``` + + Example with SSL disabled (common in Docker networks): + Add `?sslmode=disable` to the connection string when connecting within a private Docker network where SSL encryption isn't required. + ```bash + postgresql://postgres:mysecretpassword@postgres-dev:5432/testdb?sslmode=disable + ``` + +> [!NOTE] +> +> The default port `5432` is used in these examples. If you're connecting to a different PostgreSQL instance or have changed the port, update the connection string accordingly. The container name (`postgres-dev`) is resolved by Docker DNS to the container's IP address on the network. + + +### Connecting from the host (external access) + +To connect to PostgreSQL from your host machine using tools like `psql`, `pgAdmin`, `DBeaver`, or database management scripts, you need to publish PostgreSQL's port (`5432`) to the host. This allows external tools to reach the PostgreSQL container. + +#### Expose Postgres to localhost only (recommended for development) + +This binds to `127.0.0.1` so it's only reachable from your local machine, not from other devices on your network. This is the most secure option for development. + +```bash +docker run -d --name postgres-dev \ + -e POSTGRES_PASSWORD=mysecretpassword \ + -p 127.0.0.1:5432:5432 \ + -v postgres_data:/var/lib/postgresql \ + postgres:18 +``` + +Now connect from your host: + +- Host: `localhost` or `127.0.0.1` +- Port: `5432` + +If you have `psql` installed on your host: +```bash +psql -h localhost -p 5432 -U postgres +``` + +You'll be prompted for the password. Alternatively, you can use the `PGPASSWORD` environment variable: +```bash +PGPASSWORD=mysecretpassword psql -h localhost -p 5432 -U postgres +``` + +#### Connecting with PostgreSQL GUI tools + +Popular PostgreSQL GUI tools can connect using these common connection details: Host: `localhost`, Port: `5432`, User: `postgres`, Database: `postgres` (or your database name). + +- pgAdmin: A web-based PostgreSQL administration and development platform +- DBeaver: A universal database tool that supports PostgreSQL and many other databases. Select PostgreSQL as the connection type +- TablePlus: A modern, native database management tool for macOS and Windows with a clean interface + +All tools will prompt for the password you set with `POSTGRES_PASSWORD`. + +#### Expose Postgres to all network interfaces (use with caution) + +To allow connections from other devices on your network, use `-p 5432:5432` instead of `-p 127.0.0.1:5432:5432`. This binds PostgreSQL to all network interfaces on your host, making it accessible from any device that can reach your host, not just localhost. + +```bash +docker run -d --name postgres-dev \ + -e POSTGRES_PASSWORD=mysecretpassword \ + -p 5432:5432 \ + -v postgres_data:/var/lib/postgresql \ + postgres:18 +``` + +> [!WARNING] +> +> Exposing PostgreSQL to all network interfaces (`0.0.0.0:5432`) makes it accessible from any device that can reach your host. Only use this in trusted network environments or behind a firewall. For production, consider using a reverse proxy or VPN instead. + +#### PostgreSQL security considerations for external access + +When exposing PostgreSQL to external access, follow these PostgreSQL-specific security practices: + +- Avoid using the `postgres` superuser: The default `postgres` user has full database privileges. Create dedicated users with only the permissions your application needs. +- Use strong passwords: PostgreSQL passwords should be complex. Consider using environment variables or secrets management instead of `hardcoding` passwords. +- Limit network exposure: Binding to `127.0.0.1` (localhost only) is safer than exposing to all interfaces (`0.0.0.0`). +- Consider SSL/TLS: For production, configure PostgreSQL to require SSL connections. The [Advanced Configuration and Initialization](/guides/postgresql/advanced-configuration-and-initialization/) guide shows how to configure PostgreSQL settings. +- Create application-specific users: Use initialization scripts to create users with limited privileges. For example, a read-only user for reporting or a user that can only access specific databases. + +The [Advanced configuration and initialization](/guides/postgresql/advanced-configuration-and-initialization/) guide shows how to use initialization scripts to create users and roles automatically. + +### Using Docker Compose for networking + +Docker Compose automatically creates a network for your services, making networking configuration simpler. Here's an example that combines both internal and external access: + +```yaml +services: + db: + image: postgres:18 + container_name: postgres-dev + environment: + POSTGRES_PASSWORD: mysecretpassword + volumes: + - postgres_data:/var/lib/postgresql + ports: + - "127.0.0.1:5432:5432" # Expose to localhost only + networks: + - app-network + + app: + build: ./my-app + environment: + DATABASE_URL: postgresql://postgres:mysecretpassword@db:5432/mydb + networks: + - app-network + depends_on: + - db + +volumes: + postgres_data: + +networks: + app-network: + driver: bridge +``` + +In this PostgreSQL-focused setup: +- The `app` service connects to PostgreSQL using the service name (`db`) as the hostname in the connection string +- PostgreSQL is accessible from your host at `localhost:5432` for external tools +- Both services are isolated on a custom network, providing network-level security +- The `depends_on` directive ensures PostgreSQL starts before your application + +PostgreSQL connection details for the app service: +- Hostname: `db` (resolved by Docker DNS) +- Port: `5432` (PostgreSQL default port) +- Database: `mydb` (as specified in the connection string) +- User: `postgres` (or a custom user you've created) + +> [!NOTE] +> +> Docker Compose automatically creates a network for your project. Services can reach each other by service name without explicit network configuration, but defining a custom network gives you more control. For PostgreSQL, this means your application can always connect using the service name, regardless of container restarts or IP changes. + +### Troubleshooting + +This section covers common PostgreSQL connection issues and their solutions when working with Docker networking. + +#### "Could not translate host name postgres-dev" + +- Both containers must be on the same Docker network (`my-app-net`). +- Verify the network exists: `docker network ls` +- Check which network a container is on: `docker inspect postgres-dev | grep NetworkMode` +- Ensure you're using a user-defined network, not the default bridge network + +#### "Connection refused" or "could not connect to server" + +- PostgreSQL may still be initializing: PostgreSQL takes a few seconds to start and initialize the database cluster. Wait 5-10 seconds after container start and retry. +- Check if the PostgreSQL container is running: + + ```bash + docker ps --filter name=postgres-dev + ``` + +- Check PostgreSQL logs for initialization or connection errors: + + ```bash + docker logs postgres-dev + ``` + + Look for messages like "database system is ready to accept connections" to confirm PostgreSQL is fully started. + +- Verify the port mapping is correct: + + ```bash + docker port postgres-dev + ``` + + This should show `5432/tcp -> 127.0.0.1:5432` (or `0.0.0.0:5432` if bound to all interfaces). + +- Test PostgreSQL connectivity from inside the container: + + ```bash + docker exec -it postgres-dev psql -U postgres -c "SELECT version();" + ``` + + If this works but external connections fail, the issue is with port publishing, not PostgreSQL itself. + +#### "Password authentication failed" or "FATAL: password authentication failed for user" + +- Confirm the password: Verify you're using the same password set in `POSTGRES_PASSWORD` when you started the container. +- Existing volume with old credentials: If you reused an existing volume, the password from the original initialization is still in effect. The `POSTGRES_PASSWORD` environment variable only sets the password during the first database initialization. To reset: + - Remove the volume: `docker volume rm postgres_data` + - Or connect with the old password + - Or change the password after connecting: `ALTER USER postgres WITH PASSWORD 'newpassword';` +- Try connecting with password prompt: `psql -h localhost -U postgres -W` (the `-W` flag forces a password prompt) +- Use PGPASSWORD environment variable: `PGPASSWORD=mysecretpassword psql -h localhost -U postgres` +- Check PostgreSQL authentication configuration: If you've customized `pg_hba.conf`, verify the authentication method allows password authentication + +#### "Network not found" + +- Ensure the network exists before starting containers: `docker network create my-app-net` +- If using Docker Compose, the network is created automatically when you run `docker compose up` + +## Companions for PostgreSQL + +### PostgreSQL ecosystem companions: pgAdmin, PgBouncer, and performance testing + +Running a standalone PostgreSQL container is often just the beginning. What happens when thousands of connections arrive, or when you need a visual interface to manage your database? + +This is where **companion tools** come into play. These applications extend PostgreSQL with capabilities the core database engine doesn't provide natively: visual administration, connection pooling, and performance benchmarking. This guide covers how to deploy pgAdmin 4, PgBouncer, Pgpool-II, and `pgbench` in Docker, when to use each tool, and real-world benchmark results demonstrating their performance impact. + +### pgAdmin 4: Visual management platform + +pgAdmin 4 is the industry-standard open source management tool for PostgreSQL. When deployed in Docker, it typically runs in **Server Mode**, providing a multi-user web interface to manage one or more database instances. + +While you can accomplish everything from the command line using `psql`, a visual interface significantly simplifies writing complex queries, visualizing table structures, and exploring database objects. + +#### Key considerations + +When running pgAdmin in Docker, keep these points in mind: + +- **Image**: Use the official `dpage/pgadmin4` image +- **Networking**: In a Docker Compose environment, pgAdmin connects to the database using the internal service name (for example, `db:5432`) rather than `localhost` + +#### Docker Compose configuration + +To quickly deploy pgAdmin: + +```yaml +pgadmin: + image: dpage/pgadmin4:8.14 + environment: + PGADMIN_DEFAULT_EMAIL: admin@example.com + PGADMIN_DEFAULT_PASSWORD: secure_password + volumes: + - pgadmin_data:/var/lib/pgadmin + ports: + - "8080:80" +``` + +With this configuration, access the pgAdmin interface at `http://localhost:8080`. Use the email and password specified in the environment variables for initial sign in. + +> [!IMPORTANT] +> +> In production environments, pass `PGADMIN_DEFAULT_PASSWORD` as an external environment variable or use Docker secrets. Storing passwords in plain text within `docker-compose.yml` poses a security risk. + +Now that you have visual database management in place, the next challenge in production environments is handling connection load. The following section explains how to manage high-volume database traffic. + +### PgBouncer: Lightweight connection pooling + +PostgreSQL creates a new process for every client connection, which consumes significant RAM. What happens when you have 1,000 concurrent users? PgBouncer solves exactly this problem. + +PgBouncer is a lightweight proxy that pools connections, allowing thousands of applications to share a small number of actual database backends. Think of it as a traffic controller: everyone wants to pass through simultaneously, but the controller regulates the flow to prevent congestion. + +#### Pooling modes + +PgBouncer offers three distinct pooling modes: + +| Mode | Description | Use case | +|------|-------------|----------| +| **Session** | Connection assigned for entire session duration | Long-lived connections, session variables | +| **Transaction** | Connection returned after each transaction ends | Web applications, microservices (most common) | +| **Statement** | Connection returned after every SQL statement | Simple queries, no multi-statement transactions | + +#### When to use PgBouncer + +PgBouncer becomes essential when you encounter: + +- "too many connections" errors +- High memory consumption due to connection overhead +- Many short-lived connections (web applications, serverless functions) +- Need to serve thousands of clients with limited database connections + +#### Complete Docker Compose setup + +To run PostgreSQL and PgBouncer together, you need three files: `docker-compose.yml`, `pgbouncer.ini`, and `userlist.txt`. + +First, create the PgBouncer configuration file (`pgbouncer.ini`): + +```bash +[databases] +benchmark = host=postgres port=5432 dbname=benchmark user=postgres + +[pgbouncer] +listen_addr = 0.0.0.0 +listen_port = 6432 +auth_type = trust +auth_file = /etc/pgbouncer/userlist.txt +admin_users = postgres +pool_mode = transaction +max_client_conn = 1000 +default_pool_size = 50 +min_pool_size = 10 +reserve_pool_size = 10 +max_db_connections = 100 +``` + +Next, create the user authentication file (`userlist.txt`): + +```bash +"postgres" "postgres" +``` + +Finally, create the Docker Compose file (`docker-compose.yml`): + +```yaml +services: + postgres: + image: postgres:18 + container_name: postgres + environment: + POSTGRES_USER: postgres + POSTGRES_PASSWORD: postgres + POSTGRES_DB: benchmark + POSTGRES_HOST_AUTH_METHOD: trust + volumes: + - postgres_data:/var/lib/postgresql + ports: + - "5432:5432" + networks: + - pgnet + healthcheck: + test: ["CMD-SHELL", "pg_isready -U postgres"] + interval: 5s + timeout: 5s + retries: 5 + + pgbouncer: + image: percona/percona-pgbouncer:1.25.0 + container_name: pgbouncer + volumes: + - ./pgbouncer.ini:/etc/pgbouncer/pgbouncer.ini + - ./userlist.txt:/etc/pgbouncer/userlist.txt + ports: + - "6432:6432" + networks: + - pgnet + depends_on: + postgres: + condition: service_healthy + +volumes: + postgres_data: + +networks: + pgnet: + driver: bridge +``` + +Key configuration notes: + +- `PgBouncer` listens on port **6432**, avoiding confusion with the direct PostgreSQL connection on port 5432 +- The `depends_on` directive with `service_healthy` condition ensures PgBouncer starts only after PostgreSQL is ready +- `pool_mode = transaction` is the optimal choice for most web applications +- The [Percona PgBouncer image](https://hub.docker.com/r/percona/percona-pgbouncer) requires mounted configuration files (without the `:ro` flag, as the entrypoint script needs to modify them) +- This example uses `trust` authentication for simplicity. In production, configure proper SCRAM-SHA-256 authentication + +> [!NOTE] +> +> The `Percona PgBouncer` entrypoint script processes the configuration files on startup. Mount them without the read-only flag to avoid permission errors. + + + + +### `pgbench`: Performance benchmarking + +`pgbench` is a benchmarking utility included with the official PostgreSQL image. It allows you to simulate heavy workloads and verify how your Docker configuration performs under pressure. + +#### Initialize benchmark tables + +First, create the test tables. The `-s` (scale) parameter determines data size—scale factor 50 creates approximately 5 million rows: + +```bash +docker exec postgres pgbench -i -s 50 -U postgres benchmark +``` + +#### Run stress tests + +Key parameters: + +- `-c`: Number of simulated clients +- `-j`: Number of threads +- `-T`: Duration in seconds + +Test with direct PostgreSQL connection: + +```bash +docker exec postgres pgbench -h localhost -U postgres -c 50 -j 4 -T 60 benchmark +``` + +Test through PgBouncer: + +```bash +docker exec postgres pgbench -h pgbouncer -p 6432 -U postgres -c 50 -j 4 -T 60 benchmark +``` + +### Understanding benchmark results + +Does PgBouncer actually make a difference? Run the benchmarks yourself to find out. Your results will vary based on your hardware, Docker configuration, network setup, and system load. + +#### What to expect + +When you run these benchmarks, you'll observe patterns rather than specific numbers. Think of it like comparing two different routes to work: the "faster" route depends on traffic conditions, time of day, and your vehicle. + +#### Key observations + +When comparing direct connections versus PgBouncer, you'll typically notice: + +##### 1. Connection overhead differs significantly + +Direct connections require PostgreSQL to spawn a new process for each client. PgBouncer reuses existing connections. Watch the "initial connection time" metric in your results—PgBouncer often shows dramatically faster connection setup. + +##### 2. Behavior under pressure reveals the real difference + +Try increasing the client count (`-c` parameter) gradually: 50, 100, 150, 200. At some point, direct connections will fail with "too many clients already" while PgBouncer continues handling requests. This is PgBouncer's primary value: **it prevents connection exhaustion**. + +##### 3. Throughput varies by environment + +On some systems, direct connections show higher transactions per second (TPS) at low concurrency. On others, PgBouncer wins even with few clients. The difference depends on: +- CPU and memory available +- Docker networking overhead +- Disk I/O speed +- Whether connections are being rapidly opened and closed + diff --git a/content/guides/postgresql/advanced-configuration-and-initialization.md b/content/guides/postgresql/advanced-configuration-and-initialization.md deleted file mode 100644 index bd14d04a2ab7..000000000000 --- a/content/guides/postgresql/advanced-configuration-and-initialization.md +++ /dev/null @@ -1,229 +0,0 @@ ---- -title: Advanced Configuration and Initialization -weight: 20 -description: Configure PostgreSQL initialization scripts, tune performance parameters, and set timezone and locale settings for containerized deployments. -keywords: - - PostgreSQL Docker - - Docker Compose PostgreSQL - - container database - - PostgreSQL performance tuning ---- - -With persistent storage configured in the previous section, you're ready to customize PostgreSQL for real-world use. This guide covers advanced configuration techniques for running PostgreSQL in Docker containers, including automated database initialization, performance tuning, and timezone configuration. - -## Overview - -While PostgreSQL containers can be started quickly with default settings, production environments require customized configurations. This guide explains how to: - -- Automate database, schema, and user creation during container startup -- Tune PostgreSQL performance parameters for containerized workloads -- Configure timezone and locale settings - -## Initialization scripts - -The official PostgreSQL Docker image supports running initialization scripts automatically when the container starts for the first time. Any files placed in the `/docker-entrypoint-initdb.d/` directory are executed in alphabetical order. - -### How initialization works - -When the container starts, it checks whether the PostgreSQL data directory is empty. If the directory already contains data, PostgreSQL starts immediately without running any initialization. If the directory is empty, the container runs `initdb` to create a new database cluster, then executes all scripts in `/docker-entrypoint-initdb.d/` in alphabetical order before starting PostgreSQL. - -### Supported file formats - -| Format | Description | -|--------|-------------| -| `.sql` | SQL commands executed directly | -| `.sql.gz` | Gzip-compressed SQL files | -| `.sh` | Shell scripts executed with bash | - -> [!IMPORTANT] -> -> Initialization scripts only run when the PostgreSQL data directory (`/var/lib/postgresql/data`) is empty. If you mount a volume containing existing data, initialization is skipped. This behavior prevents overwriting existing databases. - -## Mounting initialization scripts - -Use Docker Compose to mount initialization scripts into the container. First, create a project directory: - -```console -$ mkdir -p postgres-project/init-db -$ cd postgres-project -``` - -Create a `compose.yaml` file: - -```yaml -services: - db: - image: postgres:18 - volumes: - - ./init-db:/docker-entrypoint-initdb.d - - postgres_data:/var/lib/postgresql - environment: - POSTGRES_PASSWORD: mysecretpassword - -volumes: - postgres_data: -``` - -All scripts in the `./init-db` directory execute when the container starts for the first time. This is great for bootstrapping databases. - -## Initialization script example - -Create a file named `init.sql` in your `init-db` directory: - -```sql -CREATE TABLE users ( - id SERIAL PRIMARY KEY, - email VARCHAR(255) UNIQUE NOT NULL, - name VARCHAR(100) NOT NULL, - created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP -); -``` - -This script runs automatically when the container starts for the first time, creating your initial database schema. - -> [!NOTE] -> -> Ensure initialization scripts have proper read permissions. If you encounter "Permission denied" errors, run `chmod 644 init-db/*.sql` to make the files readable by the container. - -## Performance tuning - -Default PostgreSQL settings are conservative to work on systems with limited resources. For production workloads, you should tune these parameters based on your container's allocated resources. - -### Method 1: Custom configuration file - -For complete control, mount a custom `postgresql.conf` file. First, extract the default configuration: - -```console -$ docker run -i --rm postgres:18 cat /usr/share/postgresql/postgresql.conf.sample > my-postgres.conf -``` - -Edit `my-postgres.conf` with your desired settings, then mount it in your Compose file: - -```yaml -services: - db: - image: postgres:18 - volumes: - - ./my-postgres.conf:/etc/postgresql/postgresql.conf - - ./init-db:/docker-entrypoint-initdb.d - - postgres_data:/var/lib/postgresql - command: postgres -c config_file=/etc/postgresql/postgresql.conf - environment: - POSTGRES_PASSWORD: mysecretpassword - -volumes: - postgres_data: -``` - -## Key configuration parameters - -The following tables list important `postgresql.conf` parameters for containerized PostgreSQL deployments. - -### Connection settings - -| Parameter | Description | Default | -|-----------|-------------|---------| -| `listen_addresses` | IP addresses to listen on | `localhost` | -| `port` | TCP port number | `5432` | -| `max_connections` | Maximum concurrent connections | `100` | - -### Memory settings - -| Parameter | Description | Recommended starting value | -|-----------|-------------|---------------------------| -| `shared_buffers` | Shared memory for caching | 25% of container memory | -| `work_mem` | Memory per query operation | 4MB - 64MB | -| `maintenance_work_mem` | Memory for VACUUM, CREATE INDEX | 64MB - 256MB | -| `effective_cache_size` | Planner's cache size estimate | 50-75% of container memory | - -#### Docker memory limits - -When tuning memory parameters, set explicit memory limits on your container using `deploy.resources.limits.memory` in Compose or `--memory` with `docker run`. Without limits, PostgreSQL sees the host's total RAM and may allocate more than intended. For example, if your container should use 4GB maximum, set `shared_buffers` to approximately 1GB (25%). - -### I/O settings - -| Parameter | Description | Recommended starting value | -|-----------|-------------|---------------------------| -| `effective_io_concurrency` | Concurrent disk I/O operations | `200` for SSDs, `2` for HDDs | - -### Timeout settings - -| Parameter | Description | Default | -|-----------|-------------|---------| -| `statement_timeout` | Max time for any statement | `0` (disabled) | -| `lock_timeout` | Max time to wait for a lock | `0` (disabled) | -| `deadlock_timeout` | Time before checking for deadlock | `1s` | -| `transaction_timeout` | Max time for a transaction | `0` (disabled) | - -> [!NOTE] -> -> Setting `shared_buffers` too high in a container can exceed kernel shared memory limits. Use no more than 25-30% of the container's memory limit. - -## Timezone and locale configuration - -Proper localization ensures timestamps and sorting behave correctly for your application's users. - -```yaml -services: - db: - image: postgres:18 - volumes: - - postgres_data:/var/lib/postgresql - - /etc/localtime:/etc/localtime:ro - - /etc/timezone:/etc/timezone:ro - environment: - POSTGRES_PASSWORD: mysecretpassword - TZ: America/New_York - -volumes: - postgres_data: -``` - -Alternatively, set the timezone using a PostgreSQL command-line parameter: - -```yaml -services: - db: - image: postgres:18 - command: ["postgres", "-c", "timezone=America/New_York"] - environment: - POSTGRES_PASSWORD: mysecretpassword -``` - -### Setting the locale - -Specify locale settings during database initialization using the `POSTGRES_INITDB_ARGS` environment variable: - -```yaml -services: - db: - image: postgres:18 - volumes: - - postgres_data:/var/lib/postgresql - environment: - POSTGRES_PASSWORD: mysecretpassword - POSTGRES_INITDB_ARGS: "--encoding=UTF8 --lc-collate=en_US.UTF-8 --lc-ctype=en_US.UTF-8" - -volumes: - postgres_data: -``` - -This affects collation (sorting) and character processing behavior. Changing this variable after database creation has no effect—it only applies during the first run when the data directory is initialized. - -## Connecting to the database - -You can interact with PostgreSQL running in a container even without `psql` installed on your host machine. - -### Interactive shell - -Open a `psql` session inside the container: - -```console -$ docker exec -it postgres-container psql -U postgres -``` - -Connect to a specific database: - -```console -$ docker exec -it postgres-container psql -U postgres -d mydb -``` \ No newline at end of file diff --git a/content/guides/postgresql/companions-for-postgresql.md b/content/guides/postgresql/companions-for-postgresql.md deleted file mode 100644 index a9db5000ec36..000000000000 --- a/content/guides/postgresql/companions-for-postgresql.md +++ /dev/null @@ -1,231 +0,0 @@ ---- -title: Companions for PostgreSQL -linkTitle: Companions for PostgreSQL -description: This module explains how to customize PostgreSQL for real-world use in Docker, covering automated database initialization, performance tuning, and timezone configuration once persistent storage is in place. -keywords: - - PostgreSQL Docker - - Docker Compose PostgreSQL - - container database -weight: 40 ---- - - -## PostgreSQL ecosystem companions: pgAdmin, PgBouncer, and performance testing - -Running a standalone PostgreSQL container is often just the beginning. What happens when thousands of connections arrive, or when you need a visual interface to manage your database? - -This is where **companion tools** come into play. These applications extend PostgreSQL with capabilities the core database engine doesn't provide natively: visual administration, connection pooling, and performance benchmarking. This guide covers how to deploy pgAdmin 4, PgBouncer, Pgpool-II, and `pgbench` in Docker, when to use each tool, and real-world benchmark results demonstrating their performance impact. - -## pgAdmin 4: Visual management platform - -pgAdmin 4 is the industry-standard open source management tool for PostgreSQL. When deployed in Docker, it typically runs in **Server Mode**, providing a multi-user web interface to manage one or more database instances. - -While you can accomplish everything from the command line using `psql`, a visual interface significantly simplifies writing complex queries, visualizing table structures, and exploring database objects. - -### Key considerations - -When running pgAdmin in Docker, keep these points in mind: - -- **Image**: Use the official `dpage/pgadmin4` image -- **Networking**: In a Docker Compose environment, pgAdmin connects to the database using the internal service name (for example, `db:5432`) rather than `localhost` - -### Docker Compose configuration - -To quickly deploy pgAdmin: - -```yaml -pgadmin: - image: dpage/pgadmin4:8.14 - environment: - PGADMIN_DEFAULT_EMAIL: admin@example.com - PGADMIN_DEFAULT_PASSWORD: secure_password - volumes: - - pgadmin_data:/var/lib/pgadmin - ports: - - "8080:80" -``` - -With this configuration, access the pgAdmin interface at `http://localhost:8080`. Use the email and password specified in the environment variables for initial sign in. - -> [!IMPORTANT] -> -> In production environments, pass `PGADMIN_DEFAULT_PASSWORD` as an external environment variable or use Docker secrets. Storing passwords in plain text within `docker-compose.yml` poses a security risk. - -Now that you have visual database management in place, the next challenge in production environments is handling connection load. The following section explains how to manage high-volume database traffic. - -## PgBouncer: Lightweight connection pooling - -PostgreSQL creates a new process for every client connection, which consumes significant RAM. What happens when you have 1,000 concurrent users? PgBouncer solves exactly this problem. - -PgBouncer is a lightweight proxy that pools connections, allowing thousands of applications to share a small number of actual database backends. Think of it as a traffic controller: everyone wants to pass through simultaneously, but the controller regulates the flow to prevent congestion. - -### Pooling modes - -PgBouncer offers three distinct pooling modes: - -| Mode | Description | Use case | -|------|-------------|----------| -| **Session** | Connection assigned for entire session duration | Long-lived connections, session variables | -| **Transaction** | Connection returned after each transaction ends | Web applications, microservices (most common) | -| **Statement** | Connection returned after every SQL statement | Simple queries, no multi-statement transactions | - -### When to use PgBouncer - -PgBouncer becomes essential when you encounter: - -- "too many connections" errors -- High memory consumption due to connection overhead -- Many short-lived connections (web applications, serverless functions) -- Need to serve thousands of clients with limited database connections - -### Complete Docker Compose setup - -To run PostgreSQL and PgBouncer together, you need three files: `docker-compose.yml`, `pgbouncer.ini`, and `userlist.txt`. - -First, create the PgBouncer configuration file (`pgbouncer.ini`): - -```bash -[databases] -benchmark = host=postgres port=5432 dbname=benchmark user=postgres - -[pgbouncer] -listen_addr = 0.0.0.0 -listen_port = 6432 -auth_type = trust -auth_file = /etc/pgbouncer/userlist.txt -admin_users = postgres -pool_mode = transaction -max_client_conn = 1000 -default_pool_size = 50 -min_pool_size = 10 -reserve_pool_size = 10 -max_db_connections = 100 -``` - -Next, create the user authentication file (`userlist.txt`): - -```bash -"postgres" "postgres" -``` - -Finally, create the Docker Compose file (`docker-compose.yml`): - -```yaml -services: - postgres: - image: postgres:18 - container_name: postgres - environment: - POSTGRES_USER: postgres - POSTGRES_PASSWORD: postgres - POSTGRES_DB: benchmark - POSTGRES_HOST_AUTH_METHOD: trust - volumes: - - postgres_data:/var/lib/postgresql - ports: - - "5432:5432" - networks: - - pgnet - healthcheck: - test: ["CMD-SHELL", "pg_isready -U postgres"] - interval: 5s - timeout: 5s - retries: 5 - - pgbouncer: - image: percona/percona-pgbouncer:1.25.0 - container_name: pgbouncer - volumes: - - ./pgbouncer.ini:/etc/pgbouncer/pgbouncer.ini - - ./userlist.txt:/etc/pgbouncer/userlist.txt - ports: - - "6432:6432" - networks: - - pgnet - depends_on: - postgres: - condition: service_healthy - -volumes: - postgres_data: - -networks: - pgnet: - driver: bridge -``` - -Key configuration notes: - -- `PgBouncer` listens on port **6432**, avoiding confusion with the direct PostgreSQL connection on port 5432 -- The `depends_on` directive with `service_healthy` condition ensures PgBouncer starts only after PostgreSQL is ready -- `pool_mode = transaction` is the optimal choice for most web applications -- The [Percona PgBouncer image](https://hub.docker.com/r/percona/percona-pgbouncer) requires mounted configuration files (without the `:ro` flag, as the entrypoint script needs to modify them) -- This example uses `trust` authentication for simplicity. In production, configure proper SCRAM-SHA-256 authentication - -> [!NOTE] -> -> The `Percona PgBouncer` entrypoint script processes the configuration files on startup. Mount them without the read-only flag to avoid permission errors. - - - - -## `pgbench`: Performance benchmarking - -`pgbench` is a benchmarking utility included with the official PostgreSQL image. It allows you to simulate heavy workloads and verify how your Docker configuration performs under pressure. - -### Initialize benchmark tables - -First, create the test tables. The `-s` (scale) parameter determines data size—scale factor 50 creates approximately 5 million rows: - -```bash -docker exec postgres pgbench -i -s 50 -U postgres benchmark -``` - -### Run stress tests - -Key parameters: - -- `-c`: Number of simulated clients -- `-j`: Number of threads -- `-T`: Duration in seconds - -Test with direct PostgreSQL connection: - -```bash -docker exec postgres pgbench -h localhost -U postgres -c 50 -j 4 -T 60 benchmark -``` - -Test through PgBouncer: - -```bash -docker exec postgres pgbench -h pgbouncer -p 6432 -U postgres -c 50 -j 4 -T 60 benchmark -``` - -## Understanding benchmark results - -Does PgBouncer actually make a difference? Run the benchmarks yourself to find out. Your results will vary based on your hardware, Docker configuration, network setup, and system load. - -### What to expect - -When you run these benchmarks, you'll observe patterns rather than specific numbers. Think of it like comparing two different routes to work: the "faster" route depends on traffic conditions, time of day, and your vehicle. - -### Key observations - -When comparing direct connections versus PgBouncer, you'll typically notice: - -#### 1. Connection overhead differs significantly - -Direct connections require PostgreSQL to spawn a new process for each client. PgBouncer reuses existing connections. Watch the "initial connection time" metric in your results—PgBouncer often shows dramatically faster connection setup. - -#### 2. Behavior under pressure reveals the real difference - -Try increasing the client count (`-c` parameter) gradually: 50, 100, 150, 200. At some point, direct connections will fail with "too many clients already" while PgBouncer continues handling requests. This is PgBouncer's primary value: **it prevents connection exhaustion**. - -#### 3. Throughput varies by environment - -On some systems, direct connections show higher transactions per second (TPS) at low concurrency. On others, PgBouncer wins even with few clients. The difference depends on: -- CPU and memory available -- Docker networking overhead -- Disk I/O speed -- Whether connections are being rapidly opened and closed - diff --git a/content/guides/postgresql/immediate-setup-and-data-persistence.md b/content/guides/postgresql/immediate-setup-and-data-persistence.md deleted file mode 100644 index 75ec251c65d5..000000000000 --- a/content/guides/postgresql/immediate-setup-and-data-persistence.md +++ /dev/null @@ -1,383 +0,0 @@ ---- -title: Immediate setup & data persistence -description: Get PostgreSQL running in Docker in under five minutes. Learn how to configure named volumes and bind mounts to persist your database across container restarts. -keywords: - - PostgreSQL Docker - - Docker Compose PostgreSQL - - container database -weight: 10 ---- - -This guide gets you from zero to a running PostgreSQL container in under five minutes, then explains how to keep your data safe across container restarts and removals. - -## Overview - -Running PostgreSQL in Docker requires understanding one critical concept: containers are ephemeral, but your data shouldn't be. This guide covers: - -- Starting PostgreSQL with a single command -- Understanding why containers lose data by default -- Configuring volumes for persistent storage -- Translating your setup to Docker Compose - -## Quick start (minimal viable container) - -> [!NOTE] -> -> [Docker Hardened Images (DHIs)](https://docs.docker.com/dhi/) are minimal, secure, and production-ready container base and application images maintained by Docker. DHIs are recommended whenever it is possible for better security. They are designed to reduce vulnerabilities and simplify compliance, freely available to everyone with no subscription required, no usage restrictions, and no vendor lock-in. - -Run PostgreSQL immediately with this single command: - -{{< tabs >}} -{{< tab name="Using DHIs" >}} - -You must authenticate to dhi.io before you can pull Docker Hardened Images. Run `docker login dhi.io` to authenticate. - -```console -docker run --rm --name postgres-dev \ - -e POSTGRES_PASSWORD=mysecretpassword \ - -p 5432:5432 \ - -d dhi.io/postgres:18 -``` - -{{< /tab >}} - -{{< tab name="Using DOIs" >}} - -```console -$ docker run --rm --name postgres-dev \ - -e POSTGRES_PASSWORD=mysecretpassword \ - -p 5432:5432 \ - -d postgres:18 -``` - -{{< /tab >}} -{{< /tabs >}} - -### Understanding the flags - -| Flag | Purpose | -|------|---------| -| `--rm` | Automatically removes the container when it stops | -| `--name postgres-dev` | Assigns a memorable name instead of a random string | -| `-e POSTGRES_PASSWORD=...` | Sets the superuser password (required) | -| `-p 5432:5432` | Maps host port 5432 to container port 5432 | -| `-d` | Runs the container in the background (detached mode) | - -Verify the container is running: - -```console -$ docker ps --filter name=postgres-dev -CONTAINER ID IMAGE COMMAND STATUS PORTS NAMES -a1b2c3d4e5f6 postgres:18 "docker-entrypoint.s…" Up 2 seconds 0.0.0.0:5432->5432/tcp postgres-dev -``` - -Connect using `psql` from inside the container: - -```console -$ docker exec -it postgres-dev psql -U postgres -psql (18.0) -Type "help" for help. - -postgres=# -``` - -You now have a working PostgreSQL instance. But there's a problem—stop this container and your data disappears. - -## The data persistence problem - -Containers use an ephemeral filesystem. When a container is removed, everything inside it, including your database files, is deleted. - -Demonstrate this yourself: - -{{< tabs >}} -{{< tab name="Using DHIs" >}} - -```console -$ docker exec postgres-dev psql -U postgres -c "CREATE DATABASE testdb;" -CREATE DATABASE - -$ docker exec postgres-dev psql -U postgres -c "\l" | grep testdb - testdb | postgres | UTF8 | libc | en_US.utf8 | en_US.utf8 | | | - -$ docker stop postgres-dev -postgres-dev - -$ docker run --rm --name postgres-dev \ - -e POSTGRES_PASSWORD=mysecretpassword \ - -p 5432:5432 \ - -d dhi.io/postgres:18 - -$ docker exec postgres-dev psql -U postgres -c "\l" | grep testdb -(no output - database is gone) -``` - -{{< /tab >}} - -{{< tab name="Using DOIs" >}} - -```console -$ docker exec postgres-dev psql -U postgres -c "CREATE DATABASE testdb;" -CREATE DATABASE - -$ docker exec postgres-dev psql -U postgres -c "\l" | grep testdb - testdb | postgres | UTF8 | libc | en_US.utf8 | en_US.utf8 | | | - -$ docker stop postgres-dev -postgres-dev - -$ docker run --rm --name postgres-dev \ - -e POSTGRES_PASSWORD=mysecretpassword \ - -p 5432:5432 \ - -d postgres:18 - -$ docker exec postgres-dev psql -U postgres -c "\l" | grep testdb -(no output - database is gone) -``` - -{{< /tab >}} -{{< /tabs >}} - -Your `testdb` database vanished because the new container started with a fresh filesystem. This is expected behavior—and exactly why volumes exist. - -## Named volumes - -Named volumes are Docker-managed storage locations that persist independently of containers. Docker handles the filesystem location, permissions, and lifecycle. - -Create a container with a named volume: - -{{< tabs >}} -{{< tab name="Using DHIs" >}} - -You must authenticate to dhi.io before you can pull Docker Hardened Images. Run `docker login dhi.io` to authenticate. - -```console -$ docker run --rm --name postgres-dev \ - -e POSTGRES_PASSWORD=mysecretpassword \ - -p 5432:5432 \ - -v postgres_data:/var/lib/postgresql \ - -d dhi.io/postgres:18 -``` - -{{< /tab >}} - -{{< tab name="Using DOIs" >}} - -```console -$ docker run --rm --name postgres-dev \ - -e POSTGRES_PASSWORD=mysecretpassword \ - -p 5432:5432 \ - -v postgres_data:/var/lib/postgresql \ - -d postgres:18 -``` - -{{< /tab >}} -{{< /tabs >}} - - -The `-v postgres_data:/var/lib/postgresql` flag mounts a named volume called `postgres_data` to PostgreSQL's data directory. If the volume doesn't exist, Docker creates it automatically. - -> [!NOTE] -> -> PostgreSQL 18+ stores data in a version-specific subdirectory under `/var/lib/postgresql`. Mounting at this level (rather than `/var/lib/postgresql/data`) allows for easier upgrades using `pg_upgrade --link`. - -### Verify persistence works - -To verify data persistence, repeat the previous test, but this time with the named volume attached in place. - -{{< tabs >}} -{{< tab name="Using DHIs" >}} - -```console -$ docker exec postgres-dev psql -U postgres -c "CREATE DATABASE testdb;" -CREATE DATABASE - -$ docker stop postgres-dev -postgres-dev - -$ docker run --rm --name postgres-dev \ - -e POSTGRES_PASSWORD=mysecretpassword \ - -p 5432:5432 \ - -v postgres_data:/var/lib/postgresql \ - -d dhi.io/postgres:18 - -$ docker exec postgres-dev psql -U postgres -c "\l" | grep testdb - testdb | postgres | UTF8 | libc | en_US.utf8 | en_US.utf8 | | | -``` - -{{< /tab >}} - -{{< tab name="Using DOIs" >}} - -```console -$ docker exec postgres-dev psql -U postgres -c "CREATE DATABASE testdb;" -CREATE DATABASE - -$ docker stop postgres-dev -postgres-dev - -$ docker run --rm --name postgres-dev \ - -e POSTGRES_PASSWORD=mysecretpassword \ - -p 5432:5432 \ - -v postgres_data:/var/lib/postgresql \ - -d postgres:18 - -$ docker exec postgres-dev psql -U postgres -c "\l" | grep testdb - testdb | postgres | UTF8 | libc | en_US.utf8 | en_US.utf8 | | | -``` - -{{< /tab >}} - -{{< /tabs >}} - -If you see `testdb` in the output, persistence works: The database survived because the volume preserved the data directory. - -### Managing volumes - -List all volumes: - -```console -$ docker volume ls --filter name=postgres_data -DRIVER VOLUME NAME -local postgres_data -``` - -Inspect a volume to see its details: - -```console -$ docker volume inspect postgres_data -[ - { - "CreatedAt": "2025-01-05T10:30:00Z", - "Driver": "local", - "Labels": null, - "Mountpoint": "/var/lib/docker/volumes/postgres_data/_data", - "Name": "postgres_data", - "Options": null, - "Scope": "local" - } -] -``` - -Remove an unused volume (warning: this deletes all data): - -```console -$ docker volume rm postgres_data -``` - -## Bind mounts (alternative) - -Bind mounts map a specific host directory to a container path. Unlike named volumes, you control exactly where data lives on the host filesystem. - -Create a directory on your host machine to store Postgres data. - -{{< tabs >}} -{{< tab name="Using DHIs" >}} - -```console -mkdir -p ~/postgres-data && sudo chown -R 999:999 ~/postgres-data -``` - -Run Postgres using a bind mount. - -```console -docker run --rm --name postgres-dev \ - -e POSTGRES_PASSWORD=mysecretpassword \ - -p 5432:5432 \ - -v ~/postgres-data:/var/lib/postgresql \ - -d dhi.io/postgres:18 -``` - -{{< /tab >}} - -{{< tab name="Using DOIs" >}} - -```console -$ mkdir -p ~/postgres-data -``` - -Run Postgres using a bind mount. - -```console -$ docker run --rm --name postgres-dev \ - -e POSTGRES_PASSWORD=mysecretpassword \ - -p 5432:5432 \ - -v ~/postgres-data:/var/lib/postgresql \ - -d postgres:18 -``` - -{{< /tab >}} -{{< /tabs >}} - -### When to use bind mounts - -Bind mounts are useful when you need direct filesystem access to the data directory for backup scripts that read files directly, when integrating with host-level monitoring tools, or when specific permission requirements exist. For most development and production scenarios, named volumes are simpler and less error-prone. - -### Common bind mount issues - -Permission errors are the most frequent problem with bind mounts. PostgreSQL runs as user `postgres` (UID 999) inside the container. If your host directory has restrictive permissions, the container fails to start. - -Check logs if the container exits immediately: - -```console -$ docker logs postgres-dev -``` - -## Docker Compose configuration - -Docker Compose captures your entire configuration in a file, making setups reproducible and easier to manage as complexity grows. - -Create a `compose.yaml` file: - -```yaml -services: - db: - image: postgres:18 - container_name: postgres-dev - environment: - POSTGRES_PASSWORD: mysecretpassword - ports: - - "5432:5432" - volumes: - - postgres_data:/var/lib/postgresql - -volumes: - postgres_data: -``` - -Start the database: - -```console -$ docker compose up -d -``` - -Stop and remove containers (volume persists): - -```console -$ docker compose down -``` - -Alternatively, you can stop, remove containers, and delete the volume: - -```console -$ docker compose down -v -``` - -This compose file becomes the foundation for adding initialization scripts, performance tuning, and companion services covered in subsequent guides. - -### Environment variables reference - -The official PostgreSQL image supports these environment variables: - -| Variable | Required | Description | -|----------|----------|-------------| -| `POSTGRES_PASSWORD` | Yes | Superuser password | -| `POSTGRES_USER` | No | Superuser name (default: `postgres`) | -| `POSTGRES_DB` | No | Default database name (default: value of `POSTGRES_USER`) | - -## Next steps - -With persistent storage configured, you're ready to customize PostgreSQL further. The next chapter of the guide covers: - -- Automated schema creation with initialization scripts -- Performance tuning for containerized workloads -- Timezone and locale configuration \ No newline at end of file diff --git a/content/guides/postgresql/networking-and-connectivity.md b/content/guides/postgresql/networking-and-connectivity.md deleted file mode 100644 index 8d62422e8fcc..000000000000 --- a/content/guides/postgresql/networking-and-connectivity.md +++ /dev/null @@ -1,339 +0,0 @@ ---- -title: Networking and connectivity -description: This module shows how to connect to PostgreSQL in Docker in two common ways; from another container (internal network) and from your host machine (external access). -keywords: - - Networking PostgreSQL Docker -weight: 30 ---- - -This guide covers two common ways to connect to PostgreSQL running in Docker: - -- Container-to-container: Connect from your application container to PostgreSQL over a private Docker network. No ports need to be exposed to the host. -- Host-to-container: Connect from your laptop or development machine using `localhost` and a published port. - -Prerequisite: This guide assumes you have PostgreSQL running with persistent storage. If you don't, follow the [Immediate Setup & Data Persistence](/guides/postgresql/immediate-setup-and-data-persistence/) guide first. - -## Internal network access (container-to-container) - -When your application runs in another container, connecting to PostgreSQL through a user-defined bridge network is the recommended approach. This setup provides automatic DNS resolution, so your application can connect to PostgreSQL using the container name as the hostname, without needing to track IP addresses. - -> [!NOTE] -> Why not use the default bridge network? While containers on the default bridge network can communicate, they can only do so by IP address. Since container IP addresses change when containers restart, this would require updating your PostgreSQL connection strings each time. User-defined bridge networks solve this by providing automatic DNS resolution, ensuring your PostgreSQL connection strings remain stable even if containers restart and receive new IP addresses. - -Here's a quick comparison: - -> [!NOTE] -> -> The following examples show the difference in approach. To actually test this, follow the steps in this guide to set up containers on the appropriate networks first. - -With the default bridge network, you'd need to find the IP address first: -```bash -# Get the container's IP address (changes on restart) -docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' postgres-dev -# Output: 172.17.0.2 - -# Then connect using that IP address from another container -# (No --network flag needed - containers default to bridge network) -docker run --rm -it \ - -e PGPASSWORD=mysecretpassword \ - postgres:18 \ - psql -h 172.17.0.2 -U postgres -``` - -With a user-defined network, you simply use the container name: -```bash -# Container name works directly - no IP lookup needed -docker run --rm -it \ - --network my-app-net \ - -e PGPASSWORD=mysecretpassword \ - postgres:18 \ - psql -h postgres-dev -U postgres -``` - -### Step 1: Create a user-defined network - -```bash -docker network create my-app-net - -# Example Output -ab7f984be43a0ca15534a9ee568716ddbe869a5875077fad3ef3192e3af7d288 - -docker network ls -# Output -ab7f984be43a my-app-net bridge local - - -``` - -### Step 2: Run PostgreSQL on that network (no port publishing) - -Notice there is no `-p 5432:5432` here. This keeps PostgreSQL internal to Docker and not accessible from the host machine, which is more secure for production environments. - -```bash -docker run -d --name postgres-dev \ - --network my-app-net \ - -e POSTGRES_PASSWORD=mysecretpassword \ - -v postgres_data:/var/lib/postgresql \ - postgres:18 - - # Output -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -6d351ed89efc postgres:18 "docker-entrypoint.s…" 9 seconds ago Up 8 seconds 5432/tcp postgres-dev - -``` - -### Step 3: Connect from another container using the Postgres container name - -You can test connectivity with a temporary `psql` client container: - -```bash -docker run --rm -it \ - --network my-app-net \ - -e PGPASSWORD=mysecretpassword \ - postgres:18 \ - psql -h postgres-dev -U postgres -``` - -Key point: `-h postgres-dev` works because Docker DNS resolves the container name on a user-defined network. The container name acts as the hostname. - -### Connection string examples - -When connecting from your application container, use these PostgreSQL connection strings: - -- PostgreSQL URI format: - This is the standard PostgreSQL connection URI format that combines all connection parameters into a single string, widely supported by PostgreSQL clients and libraries. - - ```bash - postgresql://postgres:mysecretpassword@postgres-dev:5432/postgres - ``` - - This command demonstrates passing a PostgreSQL URI connection string as an environment variable to a container, which your application can then read to connect to the database. - - Example usage in a Docker run command: - ```bash - docker run --rm -it \ - --network my-app-net \ - -e DATABASE_URL="postgresql://postgres:mysecretpassword@postgres-dev:5432/postgres" \ - alpine:latest \ - sh -c 'echo "DATABASE_URL is set to: $DATABASE_URL"' - ``` - - -- PostgreSQL connection parameters: - This format uses key-value pairs separated by spaces, which many PostgreSQL client libraries accept as an alternative to URI format. - ```bash - host=postgres-dev - port=5432 - user=postgres - password=mysecretpassword - dbname=postgres - ``` - - Example usage in application code (Python with psycopg2): - ```python - conn = psycopg2.connect( - host="postgres-dev", - port=5432, - user="postgres", - password="mysecretpassword", - dbname="postgres" - ) - ``` - -- Connecting to a specific database: - Replace the database name in the connection string to connect to a specific database instead of the default `postgres` database. - If you created a custom database (e.g., `testdb`), use: - ```bash - postgresql://postgres:mysecretpassword@postgres-dev:5432/testdb - ``` - - Example with SSL disabled (common in Docker networks): - Add `?sslmode=disable` to the connection string when connecting within a private Docker network where SSL encryption isn't required. - ```bash - postgresql://postgres:mysecretpassword@postgres-dev:5432/testdb?sslmode=disable - ``` - -> [!NOTE] -> -> The default port `5432` is used in these examples. If you're connecting to a different PostgreSQL instance or have changed the port, update the connection string accordingly. The container name (`postgres-dev`) is resolved by Docker DNS to the container's IP address on the network. - - -## Connecting from the host (external access) - -To connect to PostgreSQL from your host machine using tools like `psql`, `pgAdmin`, `DBeaver`, or database management scripts, you need to publish PostgreSQL's port (`5432`) to the host. This allows external tools to reach the PostgreSQL container. - -### Expose Postgres to localhost only (recommended for development) - -This binds to `127.0.0.1` so it's only reachable from your local machine, not from other devices on your network. This is the most secure option for development. - -```bash -docker run -d --name postgres-dev \ - -e POSTGRES_PASSWORD=mysecretpassword \ - -p 127.0.0.1:5432:5432 \ - -v postgres_data:/var/lib/postgresql \ - postgres:18 -``` - -Now connect from your host: - -- Host: `localhost` or `127.0.0.1` -- Port: `5432` - -If you have `psql` installed on your host: -```bash -psql -h localhost -p 5432 -U postgres -``` - -You'll be prompted for the password. Alternatively, you can use the `PGPASSWORD` environment variable: -```bash -PGPASSWORD=mysecretpassword psql -h localhost -p 5432 -U postgres -``` - -### Connecting with PostgreSQL GUI tools - -Popular PostgreSQL GUI tools can connect using these common connection details: Host: `localhost`, Port: `5432`, User: `postgres`, Database: `postgres` (or your database name). - -- pgAdmin: A web-based PostgreSQL administration and development platform -- DBeaver: A universal database tool that supports PostgreSQL and many other databases. Select PostgreSQL as the connection type -- TablePlus: A modern, native database management tool for macOS and Windows with a clean interface - -All tools will prompt for the password you set with `POSTGRES_PASSWORD`. - -### Expose Postgres to all network interfaces (use with caution) - -To allow connections from other devices on your network, use `-p 5432:5432` instead of `-p 127.0.0.1:5432:5432`. This binds PostgreSQL to all network interfaces on your host, making it accessible from any device that can reach your host, not just localhost. - -```bash -docker run -d --name postgres-dev \ - -e POSTGRES_PASSWORD=mysecretpassword \ - -p 5432:5432 \ - -v postgres_data:/var/lib/postgresql \ - postgres:18 -``` - -> [!WARNING] -> -> Exposing PostgreSQL to all network interfaces (`0.0.0.0:5432`) makes it accessible from any device that can reach your host. Only use this in trusted network environments or behind a firewall. For production, consider using a reverse proxy or VPN instead. - -### PostgreSQL security considerations for external access - -When exposing PostgreSQL to external access, follow these PostgreSQL-specific security practices: - -- Avoid using the `postgres` superuser: The default `postgres` user has full database privileges. Create dedicated users with only the permissions your application needs. -- Use strong passwords: PostgreSQL passwords should be complex. Consider using environment variables or secrets management instead of `hardcoding` passwords. -- Limit network exposure: Binding to `127.0.0.1` (localhost only) is safer than exposing to all interfaces (`0.0.0.0`). -- Consider SSL/TLS: For production, configure PostgreSQL to require SSL connections. The [Advanced Configuration and Initialization](/guides/postgresql/advanced-configuration-and-initialization/) guide shows how to configure PostgreSQL settings. -- Create application-specific users: Use initialization scripts to create users with limited privileges. For example, a read-only user for reporting or a user that can only access specific databases. - -The [Advanced configuration and initialization](/guides/postgresql/advanced-configuration-and-initialization/) guide shows how to use initialization scripts to create users and roles automatically. - -## Using Docker Compose for networking - -Docker Compose automatically creates a network for your services, making networking configuration simpler. Here's an example that combines both internal and external access: - -```yaml -services: - db: - image: postgres:18 - container_name: postgres-dev - environment: - POSTGRES_PASSWORD: mysecretpassword - volumes: - - postgres_data:/var/lib/postgresql - ports: - - "127.0.0.1:5432:5432" # Expose to localhost only - networks: - - app-network - - app: - build: ./my-app - environment: - DATABASE_URL: postgresql://postgres:mysecretpassword@db:5432/mydb - networks: - - app-network - depends_on: - - db - -volumes: - postgres_data: - -networks: - app-network: - driver: bridge -``` - -In this PostgreSQL-focused setup: -- The `app` service connects to PostgreSQL using the service name (`db`) as the hostname in the connection string -- PostgreSQL is accessible from your host at `localhost:5432` for external tools -- Both services are isolated on a custom network, providing network-level security -- The `depends_on` directive ensures PostgreSQL starts before your application - -PostgreSQL connection details for the app service: -- Hostname: `db` (resolved by Docker DNS) -- Port: `5432` (PostgreSQL default port) -- Database: `mydb` (as specified in the connection string) -- User: `postgres` (or a custom user you've created) - -> [!NOTE] -> -> Docker Compose automatically creates a network for your project. Services can reach each other by service name without explicit network configuration, but defining a custom network gives you more control. For PostgreSQL, this means your application can always connect using the service name, regardless of container restarts or IP changes. - -## Troubleshooting - -This section covers common PostgreSQL connection issues and their solutions when working with Docker networking. - -### "Could not translate host name postgres-dev" - -- Both containers must be on the same Docker network (`my-app-net`). -- Verify the network exists: `docker network ls` -- Check which network a container is on: `docker inspect postgres-dev | grep NetworkMode` -- Ensure you're using a user-defined network, not the default bridge network - -### "Connection refused" or "could not connect to server" - -- PostgreSQL may still be initializing: PostgreSQL takes a few seconds to start and initialize the database cluster. Wait 5-10 seconds after container start and retry. -- Check if the PostgreSQL container is running: - - ```bash - docker ps --filter name=postgres-dev - ``` - -- Check PostgreSQL logs for initialization or connection errors: - - ```bash - docker logs postgres-dev - ``` - - Look for messages like "database system is ready to accept connections" to confirm PostgreSQL is fully started. - -- Verify the port mapping is correct: - - ```bash - docker port postgres-dev - ``` - - This should show `5432/tcp -> 127.0.0.1:5432` (or `0.0.0.0:5432` if bound to all interfaces). - -- Test PostgreSQL connectivity from inside the container: - - ```bash - docker exec -it postgres-dev psql -U postgres -c "SELECT version();" - ``` - - If this works but external connections fail, the issue is with port publishing, not PostgreSQL itself. - -### "Password authentication failed" or "FATAL: password authentication failed for user" - -- Confirm the password: Verify you're using the same password set in `POSTGRES_PASSWORD` when you started the container. -- Existing volume with old credentials: If you reused an existing volume, the password from the original initialization is still in effect. The `POSTGRES_PASSWORD` environment variable only sets the password during the first database initialization. To reset: - - Remove the volume: `docker volume rm postgres_data` - - Or connect with the old password - - Or change the password after connecting: `ALTER USER postgres WITH PASSWORD 'newpassword';` -- Try connecting with password prompt: `psql -h localhost -U postgres -W` (the `-W` flag forces a password prompt) -- Use PGPASSWORD environment variable: `PGPASSWORD=mysecretpassword psql -h localhost -U postgres` -- Check PostgreSQL authentication configuration: If you've customized `pg_hba.conf`, verify the authentication method allows password authentication - -### "Network not found" - -- Ensure the network exists before starting containers: `docker network create my-app-net` -- If using Docker Compose, the network is created automatically when you run `docker compose up` diff --git a/content/guides/pre-seeding.md b/content/guides/pre-seeding.md index d08fa8d9b93d..7041f7a31300 100644 --- a/content/guides/pre-seeding.md +++ b/content/guides/pre-seeding.md @@ -4,8 +4,8 @@ linktitle: Pre-seeding database description: &desc Pre-seeding database with schema and data at startup for development environment keywords: Pre-seeding, database, postgres, container-supported development summary: *desc -tags: [app-dev, databases] params: + tags: [databases] time: 20 minutes --- @@ -123,7 +123,8 @@ Assuming that you have an existing Postgres database instance up and running, fo sampledb=# \l List of databases Name | Owner | Encoding | Collate | Ctype | ICU Locale | Locale Provider | Access privileges - -----------+----------+----------+------------+------------+------------+-----------------+----------------------- + -----------+----------+----------+------------+------------+------------+-----------------+-------------------- +--- postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 | | libc | sampledb | postgres | UTF8 | en_US.utf8 | en_US.utf8 | | libc | template0 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | | libc | =c/postgres + @@ -138,7 +139,8 @@ Assuming that you have an existing Postgres database instance up and running, fo ```console sampledb=# SELECT * FROM users; id | name | email - ----+-------+------------------- + ----+-------+---------------- +--- 1 | Alpha | alpha@example.com 2 | Beta | beta@example.com 3 | Gamma | gamma@example.com @@ -231,7 +233,8 @@ $ docker container stop postgres ```sql sampledb=# SELECT * FROM users; id | name | email - ----+-------+------------------- + ----+-------+---------------- +--- 1 | Alpha | alpha@example.com 2 | Beta | beta@example.com 3 | Gamma | gamma@example.com @@ -330,7 +333,8 @@ It is called at the end of the script to initiate the seeding process. The try.. ```console sampledb=# SELECT * FROM todos; id | task | completed - ----+----------------+----------- + ----+----------------+-------- +--- 1 | Watch netflix | f 2 | Finish podcast | f 3 | Pick up kid | f diff --git a/content/guides/python/_index.md b/content/guides/python/_index.md index 7e4c1dbfa5ad..b2f5e0133f17 100644 --- a/content/guides/python/_index.md +++ b/content/guides/python/_index.md @@ -5,17 +5,28 @@ description: Containerize Python apps using Docker keywords: Docker, getting started, Python, language summary: | This guide explains how to containerize Python applications using Docker. -toc_min: 1 -toc_max: 2 aliases: - /language/python/ - /guides/language/python/ -languages: [python] -tags: [dhi] + - /language/python/build-images/ + - /language/python/run-containers/ + - /language/python/containerize/ + - /language/python/develop/ + - /language/python/configure-ci-cd/ + - /guides/language/python/configure-ci-cd/ + - /language/python/deploy/ + - /guides/python/configure-github-actions/ + - /guides/python/containerize/ + - /guides/python/deploy/ + - /guides/python/develop/ + - /guides/python/lint-format-typing/ + - /guides/python/secure-supply-chain/ params: + tags: [cicd] time: 20 minutes --- + > **Acknowledgment** > > This guide is a community contribution. Docker would like to thank @@ -31,3 +42,2796 @@ The Python language-specific guide teaches you how to containerize a Python appl - Deploy your containerized Python application locally to Kubernetes to test and debug your deployment Start by containerizing an existing Python application. + +## Containerize a Python application + +### Prerequisites + +- You have installed the latest version of [Docker Desktop](/get-started/get-docker.md). + +### Overview + +Containerizing your application means packaging it together with its +dependencies, configuration, and runtime into a single portable unit called a +container image. Running that image creates a container, an isolated process +that behaves the same on any machine, whether it's your laptop, a CI runner, or +a production server. + +In this section, you'll containerize a simple +[FastAPI](https://fastapi.tiangolo.com) web application. You'll write a +`Dockerfile` that describes how to build the image, add a `compose.yaml` file +that defines how Docker runs your container, and then build and start the +application with one command. + +You'll use [Docker Hardened Images](/dhi/) as the base. These are minimal, +secure Python images maintained by Docker. + +### Create the application + +The sample application is a minimal FastAPI service with a single endpoint +that returns a JSON greeting. Create the following files in a new +`python-docker-example` directory. To create all the files at once, switch to +the **Scaffold script** tab in the file browser and copy the shell command. + +{{< files name="python-docker-example" >}} + +{{< file path="app.py" status="new" >}} +```python +# A minimal FastAPI application. +# The root endpoint (GET /) returns a JSON "Hello World" response. +# See https://fastapi.tiangolo.com/ for the framework reference. + +from fastapi import FastAPI + +app = FastAPI() + + +@app.get("/") +async def root(): + return {"message": "Hello World"} +``` +{{< /file >}} + +{{< file path="requirements.txt" status="new" >}} +```text +# Python package dependencies for the application, pinned for reproducible builds. +# See https://pip.pypa.io/en/stable/reference/requirements-file-format/ + +fastapi==0.115.12 +uvicorn==0.34.3 +``` +{{< /file >}} + +{{< file path=".gitignore" status="new" >}} +```text +# Files and directories that Git should ignore. This is the standard Python +# template covering bytecode, build artifacts, virtual environments, and IDE +# settings. See https://git-scm.com/docs/gitignore for syntax reference. + +# Byte-compiled / optimized / DLL files +__pycache__/ +*.py[cod] +*$py.class + +# C extensions +*.so + +# Distribution / packaging +.Python +build/ +develop-eggs/ +dist/ +downloads/ +eggs/ +.eggs/ +lib/ +lib64/ +parts/ +sdist/ +var/ +wheels/ +share/python-wheels/ +*.egg-info/ +.installed.cfg +*.egg +MANIFEST + +# Unit test / coverage reports +htmlcov/ +.tox/ +.nox/ +.coverage +.coverage.* +.cache +nosetests.xml +coverage.xml +*.cover +*.py,cover +.hypothesis/ +.pytest_cache/ +cover/ + +# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm +__pypackages__/ + +# Environments +.env +.venv +env/ +venv/ +ENV/ +env.bak/ +venv.bak/ + +# Secrets +db/password.txt +``` +{{< /file >}} + +{{< /files >}} + +If you already have Python installed and want to verify the app works before +containerizing it, you can run it locally: + +```console +$ python3 -m venv .venv +$ source .venv/bin/activate +$ pip install -r requirements.txt +$ uvicorn app:app --reload +``` + +> [!NOTE] +> +> On Windows, activate the virtual environment with `.venv\Scripts\activate` +> instead of `source .venv/bin/activate`. + +If you don't have Python installed, skip ahead to the next section. The +remaining steps run the application in a container, with no local Python +required. + +### Create the Docker assets + +Sign in to the DHI registry so Docker can pull the Python base images during +the build. The available Python images are listed in the +[catalog](https://hub.docker.com/hardened-images/catalog/dhi/python). + +```console +$ docker login dhi.io +``` + +Add the following three files to your `python-docker-example` directory. The +`Dockerfile` describes how to build the image, `compose.yaml` defines how +Docker runs the container, and `.dockerignore` keeps unwanted files out of the +build context. + +> [!TIP] +> +> [Gordon](/ai/gordon/), Docker's AI assistant, can generate Docker assets for +> your project. Ask Gordon to create a Dockerfile, Compose file, and +> `.dockerignore` tailored to your application. + +{{< files name="python-docker-example" >}} + +{{< file path="app.py" >}} +```python +# A minimal FastAPI application. +# The root endpoint (GET /) returns a JSON "Hello World" response. +# See https://fastapi.tiangolo.com/ for the framework reference. + +from fastapi import FastAPI + +app = FastAPI() + + +@app.get("/") +async def root(): + return {"message": "Hello World"} +``` +{{< /file >}} + +{{< file path="requirements.txt" >}} +```text +# Python package dependencies for the application, pinned for reproducible builds. +# See https://pip.pypa.io/en/stable/reference/requirements-file-format/ + +fastapi==0.115.12 +uvicorn==0.34.3 +``` +{{< /file >}} + +{{< file path="Dockerfile" status="new" >}} +```dockerfile +# syntax=docker/dockerfile:1 + +# Comments are provided throughout this file to help you get started. +# If you need more help, visit the Dockerfile reference guide at +# https://docs.docker.com/go/dockerfile-reference/ + +# This Dockerfile uses Docker Hardened Images (DHI) for enhanced security. +# For more information, see https://docs.docker.com/dhi/ + +# Use the dev image to build and install dependencies. +FROM dhi.io/python:3.12-dev AS builder + +WORKDIR /app + +RUN python3 -m venv /venv +ENV PATH="/venv/bin:$PATH" + +# Download dependencies as a separate step to take advantage of Docker's caching. +# Leverage a cache mount to /root/.cache/pip to speed up subsequent builds. +# Leverage a bind mount to requirements.txt to avoid having to copy them into +# this layer. +RUN --mount=type=cache,target=/root/.cache/pip \ + --mount=type=bind,source=requirements.txt,target=requirements.txt \ + pip install -r requirements.txt + +# Use the minimal runtime image. It runs as nonroot by default. +FROM dhi.io/python:3.12 + +WORKDIR /app + +COPY --from=builder /venv /venv +ENV PATH="/venv/bin:$PATH" + +# Copy the source code into the container. +COPY . . + +# Expose the port that the application listens on. +EXPOSE 8000 + +# Run the application. +CMD ["/venv/bin/python3", "-m", "uvicorn", "app:app", "--host=0.0.0.0", "--port=8000"] +``` +{{< /file >}} + +{{< file path="compose.yaml" status="new" >}} +```yaml +# Comments are provided throughout this file to help you get started. +# If you need more help, visit the Docker Compose reference guide at +# https://docs.docker.com/go/compose-spec-reference/ + +# Here the instructions define your application as a service called "server". +# This service is built from the Dockerfile in the current directory. +# You can add other services your application may depend on here, such as a +# database or a cache. For examples, see the Awesome Compose repository: +# https://github.com/docker/awesome-compose +services: + server: + build: + context: . + ports: + - 8000:8000 +``` +{{< /file >}} + +{{< file path=".dockerignore" status="new" >}} +```text +# Include any files or directories that you don't want to be copied to your +# container here (e.g., local build artifacts, temporary files, etc.). +# +# For more help, visit the .dockerignore file reference guide at +# https://docs.docker.com/go/build-context-dockerignore/ + +**/.DS_Store +**/__pycache__ +**/.venv +**/.classpath +**/.dockerignore +**/.env +**/.git +**/.gitignore +**/.project +**/.settings +**/.toolstarget +**/.vs +**/.vscode +**/*.*proj.user +**/*.dbmdl +**/*.jfm +**/bin +**/charts +**/docker-compose* +**/compose.y*ml +**/Dockerfile* +**/node_modules +**/npm-debug.log +**/obj +**/secrets.dev.yaml +**/values.dev.yaml +LICENSE +README.md +``` +{{< /file >}} + +{{< file path=".gitignore" >}} +```text +# Files and directories that Git should ignore. This is the standard Python +# template covering bytecode, build artifacts, virtual environments, and IDE +# settings. See https://git-scm.com/docs/gitignore for syntax reference. + +# Byte-compiled / optimized / DLL files +__pycache__/ +*.py[cod] +*$py.class + +# C extensions +*.so + +# Distribution / packaging +.Python +build/ +develop-eggs/ +dist/ +downloads/ +eggs/ +.eggs/ +lib/ +lib64/ +parts/ +sdist/ +var/ +wheels/ +share/python-wheels/ +*.egg-info/ +.installed.cfg +*.egg +MANIFEST + +# Unit test / coverage reports +htmlcov/ +.tox/ +.nox/ +.coverage +.coverage.* +.cache +nosetests.xml +coverage.xml +*.cover +*.py,cover +.hypothesis/ +.pytest_cache/ +cover/ + +# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm +__pypackages__/ + +# Environments +.env +.venv +env/ +venv/ +ENV/ +env.bak/ +venv.bak/ + +# Secrets +db/password.txt +``` +{{< /file >}} + +{{< /files >}} + +To learn more about each file, see the following: + +- [Dockerfile](/reference/dockerfile.md) +- [.dockerignore](/reference/dockerfile.md#dockerignore-file) +- [compose.yaml](/reference/compose-file/_index.md) + +### Run the application + +Inside the `python-docker-example` directory, run the following command in a +terminal. + +```console +$ docker compose up --build +``` + +Open a browser and view the application at [http://localhost:8000](http://localhost:8000). You should see a simple FastAPI application. + +In the terminal, press `ctrl`+`c` to stop the application. + +#### Run the application in the background + +You can run the application detached from the terminal by adding the `-d` +option. Inside the `python-docker-example` directory, run the following command +in a terminal. + +```console +$ docker compose up --build -d +``` + +Open a browser and view the application at [http://localhost:8000](http://localhost:8000). + +To see the OpenAPI docs you can go to [http://localhost:8000/docs](http://localhost:8000/docs). + +You should see a simple FastAPI application. + +In the terminal, run the following command to stop the application. + +```console +$ docker compose down +``` + +For more information about Compose commands, see the [Compose CLI +reference](/reference/cli/docker/compose/). + +### Summary + +In this section, you learned how you can containerize and run your Python +application using Docker. + +Related information: + +- [Docker Hardened Images](/dhi/) +- [Dockerfile reference](/reference/dockerfile.md) +- [Multi-stage builds](/manuals/build/building/multi-stage.md) +- [Docker Compose overview](/manuals/compose/_index.md) + +### Next steps + +In the next section, you'll take a look at how to set up a local development environment using Docker containers. + +## Use containers for Python development + +### Prerequisites + +Complete [Containerize a Python application](containerize.md). + +### Overview + +Once your application runs in a container, the next step is making the +container loop part of your everyday development workflow. Code changes should +show up quickly, and services your app depends on, like databases, should run +right alongside it. + +In this section, you'll extend the project from the previous topic by adding a +PostgreSQL database service to your `compose.yaml`, persisting the database +data in a named volume, and enabling Compose Watch so that changes you save in +your editor are picked up by the running container without a manual rebuild. + +### Update the application + +You'll update your application to connect to a PostgreSQL database. Continue +working in your `python-docker-example` directory. + +Replace `app.py` and `requirements.txt`, and add a new `config.py` file with the +following contents. + +> [!NOTE] +> +> The application won't run yet after this step. It tries to connect to a +> PostgreSQL database that doesn't exist. The next two sections add the +> database service and the Docker configuration needed to run everything +> together. + +{{< files name="python-docker-example" >}} + +{{< file path="app.py" status="modified" >}} +```python +# FastAPI application backed by a PostgreSQL database via SQLModel. +# The FastAPI lifespan handler creates database tables at startup. +# Endpoints: GET / (greeting), POST /heroes/ (create), GET /heroes/ (list). +# See https://fastapi.tiangolo.com/ and https://sqlmodel.tiangolo.com/ + +from collections.abc import AsyncGenerator, Sequence +from contextlib import asynccontextmanager + +from fastapi import FastAPI +from sqlmodel import Field, Session, SQLModel, create_engine, select + +from config import settings + + +class Hero(SQLModel, table=True): + id: int | None = Field(default=None, primary_key=True) + name: str = Field(index=True) + secret_name: str + age: int | None = Field(default=None, index=True) + + +engine = create_engine(str(settings.SQLALCHEMY_DATABASE_URI)) + + +def create_db_and_tables() -> None: + SQLModel.metadata.create_all(engine) + + +@asynccontextmanager +async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]: + create_db_and_tables() + yield + + +app = FastAPI(lifespan=lifespan) + + +@app.get("/") +def hello() -> str: + return "Hello, Docker!" + + +@app.post("/heroes/") +def create_hero(hero: Hero) -> Hero: + with Session(engine) as session: + session.add(hero) + session.commit() + session.refresh(hero) + return hero + + +@app.get("/heroes/") +def read_heroes() -> Sequence[Hero]: + with Session(engine) as session: + heroes = session.exec(select(Hero)).all() + return heroes +``` +{{< /file >}} + +{{< file path="config.py" status="new" >}} +```python +# Pydantic settings that read PostgreSQL connection details from the +# environment. Supports a password file (Docker secrets) via +# POSTGRES_PASSWORD_FILE in addition to POSTGRES_PASSWORD. +# See https://docs.pydantic.dev/latest/concepts/pydantic_settings/ + +import os +from typing import Any + +from pydantic import ( + PostgresDsn, + computed_field, + field_validator, + model_validator, +) +from pydantic_core import MultiHostUrl +from pydantic_settings import BaseSettings + + +class Settings(BaseSettings): + POSTGRES_SERVER: str + POSTGRES_PORT: int = 5432 + POSTGRES_USER: str + POSTGRES_PASSWORD: str | None = None + POSTGRES_PASSWORD_FILE: str | None = None + POSTGRES_DB: str + + @model_validator(mode="before") + @classmethod + def check_postgres_password(cls, data: Any) -> Any: + """Validate that either POSTGRES_PASSWORD or POSTGRES_PASSWORD_FILE is set.""" + if isinstance(data, dict): + password_file: str | None = data.get("POSTGRES_PASSWORD_FILE") # type: ignore + password: str | None = data.get("POSTGRES_PASSWORD") # type: ignore + if password_file is None and password is None: + raise ValueError( + "At least one of POSTGRES_PASSWORD_FILE and POSTGRES_PASSWORD must be set." + ) + return data # type: ignore + + @field_validator("POSTGRES_PASSWORD_FILE", mode="before") + @classmethod + def read_password_from_file(cls, v: str | None) -> str | None: + if v is not None: + file_path = v + if os.path.exists(file_path): + with open(file_path) as file: + return file.read().strip() + raise ValueError(f"Password file {file_path} does not exist.") + return v + + @computed_field + @property + def SQLALCHEMY_DATABASE_URI(self) -> PostgresDsn: + url = MultiHostUrl.build( + scheme="postgresql+psycopg", + username=self.POSTGRES_USER, + password=self.POSTGRES_PASSWORD + if self.POSTGRES_PASSWORD + else self.POSTGRES_PASSWORD_FILE, + host=self.POSTGRES_SERVER, + port=self.POSTGRES_PORT, + path=self.POSTGRES_DB, + ) + return PostgresDsn(url) + + +settings = Settings() # type: ignore +``` +{{< /file >}} + +{{< file path="requirements.txt" status="modified" hl_lines="5-7" >}} +```text +# Python package dependencies for the application, pinned for reproducible builds. +# See https://pip.pypa.io/en/stable/reference/requirements-file-format/ + +fastapi==0.115.12 +sqlmodel==0.0.24 +psycopg[binary]==3.2.9 +pydantic-settings==2.9.1 +uvicorn==0.34.3 +``` +{{< /file >}} + +{{< file path="Dockerfile" >}} +```dockerfile +# syntax=docker/dockerfile:1 + +# Comments are provided throughout this file to help you get started. +# If you need more help, visit the Dockerfile reference guide at +# https://docs.docker.com/go/dockerfile-reference/ + +# This Dockerfile uses Docker Hardened Images (DHI) for enhanced security. +# For more information, see https://docs.docker.com/dhi/ + +# Use the dev image to build and install dependencies. +FROM dhi.io/python:3.12-dev AS builder + +WORKDIR /app + +RUN python3 -m venv /venv +ENV PATH="/venv/bin:$PATH" + +# Download dependencies as a separate step to take advantage of Docker's caching. +# Leverage a cache mount to /root/.cache/pip to speed up subsequent builds. +# Leverage a bind mount to requirements.txt to avoid having to copy them into +# this layer. +RUN --mount=type=cache,target=/root/.cache/pip \ + --mount=type=bind,source=requirements.txt,target=requirements.txt \ + pip install -r requirements.txt + +# Use the minimal runtime image. It runs as nonroot by default. +FROM dhi.io/python:3.12 + +WORKDIR /app + +COPY --from=builder /venv /venv +ENV PATH="/venv/bin:$PATH" + +# Copy the source code into the container. +COPY . . + +# Expose the port that the application listens on. +EXPOSE 8000 + +# Run the application. +CMD ["/venv/bin/python3", "-m", "uvicorn", "app:app", "--host=0.0.0.0", "--port=8000"] +``` +{{< /file >}} + +{{< file path="compose.yaml" >}} +```yaml +# Comments are provided throughout this file to help you get started. +# If you need more help, visit the Docker Compose reference guide at +# https://docs.docker.com/go/compose-spec-reference/ + +# Here the instructions define your application as a service called "server". +# This service is built from the Dockerfile in the current directory. +# You can add other services your application may depend on here, such as a +# database or a cache. For examples, see the Awesome Compose repository: +# https://github.com/docker/awesome-compose +services: + server: + build: + context: . + ports: + - 8000:8000 +``` +{{< /file >}} + +{{< file path=".dockerignore" >}} +```text +# Include any files or directories that you don't want to be copied to your +# container here (e.g., local build artifacts, temporary files, etc.). +# +# For more help, visit the .dockerignore file reference guide at +# https://docs.docker.com/go/build-context-dockerignore/ + +**/.DS_Store +**/__pycache__ +**/.venv +**/.classpath +**/.dockerignore +**/.env +**/.git +**/.gitignore +**/.project +**/.settings +**/.toolstarget +**/.vs +**/.vscode +**/*.*proj.user +**/*.dbmdl +**/*.jfm +**/bin +**/charts +**/docker-compose* +**/compose.y*ml +**/Dockerfile* +**/node_modules +**/npm-debug.log +**/obj +**/secrets.dev.yaml +**/values.dev.yaml +LICENSE +README.md +``` +{{< /file >}} + +{{< file path=".gitignore" >}} +```text +# Files and directories that Git should ignore. This is the standard Python +# template covering bytecode, build artifacts, virtual environments, and IDE +# settings. See https://git-scm.com/docs/gitignore for syntax reference. + +# Byte-compiled / optimized / DLL files +__pycache__/ +*.py[cod] +*$py.class + +# C extensions +*.so + +# Distribution / packaging +.Python +build/ +develop-eggs/ +dist/ +downloads/ +eggs/ +.eggs/ +lib/ +lib64/ +parts/ +sdist/ +var/ +wheels/ +share/python-wheels/ +*.egg-info/ +.installed.cfg +*.egg +MANIFEST + +# Unit test / coverage reports +htmlcov/ +.tox/ +.nox/ +.coverage +.coverage.* +.cache +nosetests.xml +coverage.xml +*.cover +*.py,cover +.hypothesis/ +.pytest_cache/ +cover/ + +# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm +__pypackages__/ + +# Environments +.env +.venv +env/ +venv/ +ENV/ +env.bak/ +venv.bak/ + +# Secrets +db/password.txt +``` +{{< /file >}} + +{{< /files >}} + +### Update Docker assets + +Replace `Dockerfile` and `compose.yaml` with the following. + +{{< files name="python-docker-example" >}} + +{{< file path="app.py" >}} +```python +# FastAPI application backed by a PostgreSQL database via SQLModel. +# The FastAPI lifespan handler creates database tables at startup. +# Endpoints: GET / (greeting), POST /heroes/ (create), GET /heroes/ (list). +# See https://fastapi.tiangolo.com/ and https://sqlmodel.tiangolo.com/ + +from collections.abc import AsyncGenerator, Sequence +from contextlib import asynccontextmanager + +from fastapi import FastAPI +from sqlmodel import Field, Session, SQLModel, create_engine, select + +from config import settings + + +class Hero(SQLModel, table=True): + id: int | None = Field(default=None, primary_key=True) + name: str = Field(index=True) + secret_name: str + age: int | None = Field(default=None, index=True) + + +engine = create_engine(str(settings.SQLALCHEMY_DATABASE_URI)) + + +def create_db_and_tables() -> None: + SQLModel.metadata.create_all(engine) + + +@asynccontextmanager +async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]: + create_db_and_tables() + yield + + +app = FastAPI(lifespan=lifespan) + + +@app.get("/") +def hello() -> str: + return "Hello, Docker!" + + +@app.post("/heroes/") +def create_hero(hero: Hero) -> Hero: + with Session(engine) as session: + session.add(hero) + session.commit() + session.refresh(hero) + return hero + + +@app.get("/heroes/") +def read_heroes() -> Sequence[Hero]: + with Session(engine) as session: + heroes = session.exec(select(Hero)).all() + return heroes +``` +{{< /file >}} + +{{< file path="config.py" >}} +```python +# Pydantic settings that read PostgreSQL connection details from the +# environment. Supports a password file (Docker secrets) via +# POSTGRES_PASSWORD_FILE in addition to POSTGRES_PASSWORD. +# See https://docs.pydantic.dev/latest/concepts/pydantic_settings/ + +import os +from typing import Any + +from pydantic import ( + PostgresDsn, + computed_field, + field_validator, + model_validator, +) +from pydantic_core import MultiHostUrl +from pydantic_settings import BaseSettings + + +class Settings(BaseSettings): + POSTGRES_SERVER: str + POSTGRES_PORT: int = 5432 + POSTGRES_USER: str + POSTGRES_PASSWORD: str | None = None + POSTGRES_PASSWORD_FILE: str | None = None + POSTGRES_DB: str + + @model_validator(mode="before") + @classmethod + def check_postgres_password(cls, data: Any) -> Any: + """Validate that either POSTGRES_PASSWORD or POSTGRES_PASSWORD_FILE is set.""" + if isinstance(data, dict): + password_file: str | None = data.get("POSTGRES_PASSWORD_FILE") # type: ignore + password: str | None = data.get("POSTGRES_PASSWORD") # type: ignore + if password_file is None and password is None: + raise ValueError( + "At least one of POSTGRES_PASSWORD_FILE and POSTGRES_PASSWORD must be set." + ) + return data # type: ignore + + @field_validator("POSTGRES_PASSWORD_FILE", mode="before") + @classmethod + def read_password_from_file(cls, v: str | None) -> str | None: + if v is not None: + file_path = v + if os.path.exists(file_path): + with open(file_path) as file: + return file.read().strip() + raise ValueError(f"Password file {file_path} does not exist.") + return v + + @computed_field + @property + def SQLALCHEMY_DATABASE_URI(self) -> PostgresDsn: + url = MultiHostUrl.build( + scheme="postgresql+psycopg", + username=self.POSTGRES_USER, + password=self.POSTGRES_PASSWORD + if self.POSTGRES_PASSWORD + else self.POSTGRES_PASSWORD_FILE, + host=self.POSTGRES_SERVER, + port=self.POSTGRES_PORT, + path=self.POSTGRES_DB, + ) + return PostgresDsn(url) + + +settings = Settings() # type: ignore +``` +{{< /file >}} + +{{< file path="requirements.txt" >}} +```text +# Python package dependencies for the application, pinned for reproducible builds. +# See https://pip.pypa.io/en/stable/reference/requirements-file-format/ + +fastapi==0.115.12 +sqlmodel==0.0.24 +psycopg[binary]==3.2.9 +pydantic-settings==2.9.1 +uvicorn==0.34.3 +``` +{{< /file >}} + +{{< file path="Dockerfile" status="modified" hl_lines="11,27-34,37,45" >}} +```dockerfile +# syntax=docker/dockerfile:1 + +# Comments are provided throughout this file to help you get started. +# If you need more help, visit the Dockerfile reference guide at +# https://docs.docker.com/go/dockerfile-reference/ + +# This Dockerfile uses Docker Hardened Images (DHI) for enhanced security. +# For more information, see https://docs.docker.com/dhi/ + +# Use the dev image to build and install dependencies. +# The builder stage is also used directly in development (see compose.yaml). +FROM dhi.io/python:3.12-dev AS builder + +WORKDIR /app + +RUN python3 -m venv /venv +ENV PATH="/venv/bin:$PATH" + +# Download dependencies as a separate step to take advantage of Docker's caching. +# Leverage a cache mount to /root/.cache/pip to speed up subsequent builds. +# Leverage a bind mount to requirements.txt to avoid having to copy them +# into this layer. +RUN --mount=type=cache,target=/root/.cache/pip \ + --mount=type=bind,source=requirements.txt,target=requirements.txt \ + pip install -r requirements.txt + +# Copy the source code into the container. +COPY . . + +# Expose the port that the application listens on. +EXPOSE 8000 + +# Run the application. +CMD ["/venv/bin/python3", "-m", "uvicorn", "app:app", "--host=0.0.0.0", "--port=8000"] + + +# Use the minimal runtime image for production. It runs as nonroot by default. +FROM dhi.io/python:3.12 + +WORKDIR /app + +COPY --from=builder /venv /venv +ENV PATH="/venv/bin:$PATH" + +COPY --from=builder /app . + +EXPOSE 8000 + +CMD ["/venv/bin/python3", "-m", "uvicorn", "app:app", "--host=0.0.0.0", "--port=8000"] +``` +{{< /file >}} + +{{< file path="compose.yaml" status="modified" hl_lines="8" >}} +```yaml +services: + # Application service. The `target: builder` line builds the development + # image (includes a shell and tools); the production stage of the + # Dockerfile is unused in development. + server: + build: + context: . + target: builder + ports: + - 8000:8000 +``` +{{< /file >}} + +{{< file path=".dockerignore" >}} +```text +# Include any files or directories that you don't want to be copied to your +# container here (e.g., local build artifacts, temporary files, etc.). +# +# For more help, visit the .dockerignore file reference guide at +# https://docs.docker.com/go/build-context-dockerignore/ + +**/.DS_Store +**/__pycache__ +**/.venv +**/.classpath +**/.dockerignore +**/.env +**/.git +**/.gitignore +**/.project +**/.settings +**/.toolstarget +**/.vs +**/.vscode +**/*.*proj.user +**/*.dbmdl +**/*.jfm +**/bin +**/charts +**/docker-compose* +**/compose.y*ml +**/Dockerfile* +**/node_modules +**/npm-debug.log +**/obj +**/secrets.dev.yaml +**/values.dev.yaml +LICENSE +README.md +``` +{{< /file >}} + +{{< file path=".gitignore" >}} +```text +# Files and directories that Git should ignore. This is the standard Python +# template covering bytecode, build artifacts, virtual environments, and IDE +# settings. See https://git-scm.com/docs/gitignore for syntax reference. + +# Byte-compiled / optimized / DLL files +__pycache__/ +*.py[cod] +*$py.class + +# C extensions +*.so + +# Distribution / packaging +.Python +build/ +develop-eggs/ +dist/ +downloads/ +eggs/ +.eggs/ +lib/ +lib64/ +parts/ +sdist/ +var/ +wheels/ +share/python-wheels/ +*.egg-info/ +.installed.cfg +*.egg +MANIFEST + +# Unit test / coverage reports +htmlcov/ +.tox/ +.nox/ +.coverage +.coverage.* +.cache +nosetests.xml +coverage.xml +*.cover +*.py,cover +.hypothesis/ +.pytest_cache/ +cover/ + +# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm +__pypackages__/ + +# Environments +.env +.venv +env/ +venv/ +ENV/ +env.bak/ +venv.bak/ + +# Secrets +db/password.txt +``` +{{< /file >}} + +{{< /files >}} + +#### About these changes + +The `Dockerfile` builder stage now includes `COPY . .` and a `CMD` +instruction, which makes it directly runnable. This lets Compose target the +builder stage during development without rebuilding the production stage. The +production stage at the bottom is unchanged and still produces a minimal, +nonroot runtime image for shipping. + +In `compose.yaml`, the new `target: builder` line tells Compose to build and +run the builder stage of the Dockerfile during development. Unlike the minimal +production image, the development image includes a shell and additional tools +that make debugging easier. If you need a shell in a running production +container, use [Docker Debug](/reference/cli/docker/debug/) instead. + +### Add a local database and persist data + +You can use containers to set up local services, like a database. In this +section, you'll update the `compose.yaml` file to define a database service +and a volume to persist data, and add a `db/password.txt` file that holds the +database password. + +{{< files name="python-docker-example" >}} + +{{< file path="app.py" >}} +```python +# FastAPI application backed by a PostgreSQL database via SQLModel. +# The FastAPI lifespan handler creates database tables at startup. +# Endpoints: GET / (greeting), POST /heroes/ (create), GET /heroes/ (list). +# See https://fastapi.tiangolo.com/ and https://sqlmodel.tiangolo.com/ + +from collections.abc import AsyncGenerator, Sequence +from contextlib import asynccontextmanager + +from fastapi import FastAPI +from sqlmodel import Field, Session, SQLModel, create_engine, select + +from config import settings + + +class Hero(SQLModel, table=True): + id: int | None = Field(default=None, primary_key=True) + name: str = Field(index=True) + secret_name: str + age: int | None = Field(default=None, index=True) + + +engine = create_engine(str(settings.SQLALCHEMY_DATABASE_URI)) + + +def create_db_and_tables() -> None: + SQLModel.metadata.create_all(engine) + + +@asynccontextmanager +async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]: + create_db_and_tables() + yield + + +app = FastAPI(lifespan=lifespan) + + +@app.get("/") +def hello() -> str: + return "Hello, Docker!" + + +@app.post("/heroes/") +def create_hero(hero: Hero) -> Hero: + with Session(engine) as session: + session.add(hero) + session.commit() + session.refresh(hero) + return hero + + +@app.get("/heroes/") +def read_heroes() -> Sequence[Hero]: + with Session(engine) as session: + heroes = session.exec(select(Hero)).all() + return heroes +``` +{{< /file >}} + +{{< file path="config.py" >}} +```python +# Pydantic settings that read PostgreSQL connection details from the +# environment. Supports a password file (Docker secrets) via +# POSTGRES_PASSWORD_FILE in addition to POSTGRES_PASSWORD. +# See https://docs.pydantic.dev/latest/concepts/pydantic_settings/ + +import os +from typing import Any + +from pydantic import ( + PostgresDsn, + computed_field, + field_validator, + model_validator, +) +from pydantic_core import MultiHostUrl +from pydantic_settings import BaseSettings + + +class Settings(BaseSettings): + POSTGRES_SERVER: str + POSTGRES_PORT: int = 5432 + POSTGRES_USER: str + POSTGRES_PASSWORD: str | None = None + POSTGRES_PASSWORD_FILE: str | None = None + POSTGRES_DB: str + + @model_validator(mode="before") + @classmethod + def check_postgres_password(cls, data: Any) -> Any: + """Validate that either POSTGRES_PASSWORD or POSTGRES_PASSWORD_FILE is set.""" + if isinstance(data, dict): + password_file: str | None = data.get("POSTGRES_PASSWORD_FILE") # type: ignore + password: str | None = data.get("POSTGRES_PASSWORD") # type: ignore + if password_file is None and password is None: + raise ValueError( + "At least one of POSTGRES_PASSWORD_FILE and POSTGRES_PASSWORD must be set." + ) + return data # type: ignore + + @field_validator("POSTGRES_PASSWORD_FILE", mode="before") + @classmethod + def read_password_from_file(cls, v: str | None) -> str | None: + if v is not None: + file_path = v + if os.path.exists(file_path): + with open(file_path) as file: + return file.read().strip() + raise ValueError(f"Password file {file_path} does not exist.") + return v + + @computed_field + @property + def SQLALCHEMY_DATABASE_URI(self) -> PostgresDsn: + url = MultiHostUrl.build( + scheme="postgresql+psycopg", + username=self.POSTGRES_USER, + password=self.POSTGRES_PASSWORD + if self.POSTGRES_PASSWORD + else self.POSTGRES_PASSWORD_FILE, + host=self.POSTGRES_SERVER, + port=self.POSTGRES_PORT, + path=self.POSTGRES_DB, + ) + return PostgresDsn(url) + + +settings = Settings() # type: ignore +``` +{{< /file >}} + +{{< file path="requirements.txt" >}} +```text +# Python package dependencies for the application, pinned for reproducible builds. +# See https://pip.pypa.io/en/stable/reference/requirements-file-format/ + +fastapi==0.115.12 +sqlmodel==0.0.24 +psycopg[binary]==3.2.9 +pydantic-settings==2.9.1 +uvicorn==0.34.3 +``` +{{< /file >}} + +{{< file path="Dockerfile" >}} +```dockerfile +# syntax=docker/dockerfile:1 + +# Comments are provided throughout this file to help you get started. +# If you need more help, visit the Dockerfile reference guide at +# https://docs.docker.com/go/dockerfile-reference/ + +# This Dockerfile uses Docker Hardened Images (DHI) for enhanced security. +# For more information, see https://docs.docker.com/dhi/ + +# Use the dev image to build and install dependencies. +# The builder stage is also used directly in development (see compose.yaml). +FROM dhi.io/python:3.12-dev AS builder + +WORKDIR /app + +RUN python3 -m venv /venv +ENV PATH="/venv/bin:$PATH" + +# Download dependencies as a separate step to take advantage of Docker's caching. +# Leverage a cache mount to /root/.cache/pip to speed up subsequent builds. +# Leverage a bind mount to requirements.txt to avoid having to copy them +# into this layer. +RUN --mount=type=cache,target=/root/.cache/pip \ + --mount=type=bind,source=requirements.txt,target=requirements.txt \ + pip install -r requirements.txt + +# Copy the source code into the container. +COPY . . + +# Expose the port that the application listens on. +EXPOSE 8000 + +# Run the application. +CMD ["/venv/bin/python3", "-m", "uvicorn", "app:app", "--host=0.0.0.0", "--port=8000"] + + +# Use the minimal runtime image for production. It runs as nonroot by default. +FROM dhi.io/python:3.12 + +WORKDIR /app + +COPY --from=builder /venv /venv +ENV PATH="/venv/bin:$PATH" + +COPY --from=builder /app . + +EXPOSE 8000 + +CMD ["/venv/bin/python3", "-m", "uvicorn", "app:app", "--host=0.0.0.0", "--port=8000"] +``` +{{< /file >}} + +{{< file path="compose.yaml" status="modified" hl_lines="11-46" >}} +```yaml +services: + # Application service. The `target: builder` line builds the development + # image (includes a shell and tools); the production stage of the + # Dockerfile is unused in development. + server: + build: + context: . + target: builder + ports: + - 8000:8000 + environment: + - POSTGRES_SERVER=db + - POSTGRES_USER=postgres + - POSTGRES_DB=example + - POSTGRES_PASSWORD_FILE=/run/secrets/db-password + depends_on: + db: + condition: service_healthy + secrets: + - db-password + # Database service. Reads the password from a Docker secret mounted at + # /run/secrets/db-password. Compose waits for the healthcheck to pass + # before starting the server, via the server's depends_on. + db: + image: dhi.io/postgres:18 + restart: always + user: postgres + secrets: + - db-password + volumes: + - db-data:/var/lib/postgresql + environment: + - POSTGRES_DB=example + - POSTGRES_PASSWORD_FILE=/run/secrets/db-password + expose: + - 5432 + healthcheck: + test: ["CMD", "pg_isready"] + interval: 10s + timeout: 5s + retries: 5 +volumes: + db-data: +secrets: + db-password: + file: db/password.txt +``` +{{< /file >}} + +{{< file path="db/password.txt" status="new" >}} +```text +mysecretpassword +``` +{{< /file >}} + +{{< file path=".dockerignore" >}} +```text +# Include any files or directories that you don't want to be copied to your +# container here (e.g., local build artifacts, temporary files, etc.). +# +# For more help, visit the .dockerignore file reference guide at +# https://docs.docker.com/go/build-context-dockerignore/ + +**/.DS_Store +**/__pycache__ +**/.venv +**/.classpath +**/.dockerignore +**/.env +**/.git +**/.gitignore +**/.project +**/.settings +**/.toolstarget +**/.vs +**/.vscode +**/*.*proj.user +**/*.dbmdl +**/*.jfm +**/bin +**/charts +**/docker-compose* +**/compose.y*ml +**/Dockerfile* +**/node_modules +**/npm-debug.log +**/obj +**/secrets.dev.yaml +**/values.dev.yaml +LICENSE +README.md +``` +{{< /file >}} + +{{< file path=".gitignore" >}} +```text +# Files and directories that Git should ignore. This is the standard Python +# template covering bytecode, build artifacts, virtual environments, and IDE +# settings. See https://git-scm.com/docs/gitignore for syntax reference. + +# Byte-compiled / optimized / DLL files +__pycache__/ +*.py[cod] +*$py.class + +# C extensions +*.so + +# Distribution / packaging +.Python +build/ +develop-eggs/ +dist/ +downloads/ +eggs/ +.eggs/ +lib/ +lib64/ +parts/ +sdist/ +var/ +wheels/ +share/python-wheels/ +*.egg-info/ +.installed.cfg +*.egg +MANIFEST + +# Unit test / coverage reports +htmlcov/ +.tox/ +.nox/ +.coverage +.coverage.* +.cache +nosetests.xml +coverage.xml +*.cover +*.py,cover +.hypothesis/ +.pytest_cache/ +cover/ + +# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm +__pypackages__/ + +# Environments +.env +.venv +env/ +venv/ +ENV/ +env.bak/ +venv.bak/ + +# Secrets +db/password.txt +``` +{{< /file >}} + +{{< /files >}} + +> [!NOTE] +> +> To learn more about the instructions in the Compose file, see [Compose file +> reference](/reference/compose-file/). + +Now, run the following `docker compose up` command to start your application. + +```console +$ docker compose up --build +``` + +Now test your API endpoint. Open a new terminal then make a request to the server using the curl commands: + +Create an object with a POST request: + +```console +$ curl -X 'POST' \ + 'http://localhost:8000/heroes/' \ + -H 'accept: application/json' \ + -H 'Content-Type: application/json' \ + -d '{ + "id": 1, + "name": "my hero", + "secret_name": "austing", + "age": 12 +}' +``` + +You should receive the following response: + +```json +{ + "age": 12, + "id": 1, + "name": "my hero", + "secret_name": "austing" +} +``` + +Now make a GET request: + +```console +$ curl -X 'GET' \ + 'http://localhost:8000/heroes/' \ + -H 'accept: application/json' +``` + +You should receive the same response as above because it's the only object in the database. + +```json +{ + "age": 12, + "id": 1, + "name": "my hero", + "secret_name": "austing" +} +``` + +Press `ctrl+c` in the terminal to stop your application. + +### Automatically update services + +Use Compose Watch to automatically update your running Compose services as you +edit and save your code. For more details about Compose Watch, see [Use Compose +Watch](/manuals/compose/how-tos/file-watch.md). + +Open your `compose.yaml` file in an IDE or text editor and add the highlighted +Compose Watch instructions. + +{{< files name="python-docker-example" >}} + +{{< file path="app.py" >}} +```python +# FastAPI application backed by a PostgreSQL database via SQLModel. +# The FastAPI lifespan handler creates database tables at startup. +# Endpoints: GET / (greeting), POST /heroes/ (create), GET /heroes/ (list). +# See https://fastapi.tiangolo.com/ and https://sqlmodel.tiangolo.com/ + +from collections.abc import AsyncGenerator, Sequence +from contextlib import asynccontextmanager + +from fastapi import FastAPI +from sqlmodel import Field, Session, SQLModel, create_engine, select + +from config import settings + + +class Hero(SQLModel, table=True): + id: int | None = Field(default=None, primary_key=True) + name: str = Field(index=True) + secret_name: str + age: int | None = Field(default=None, index=True) + + +engine = create_engine(str(settings.SQLALCHEMY_DATABASE_URI)) + + +def create_db_and_tables() -> None: + SQLModel.metadata.create_all(engine) + + +@asynccontextmanager +async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]: + create_db_and_tables() + yield + + +app = FastAPI(lifespan=lifespan) + + +@app.get("/") +def hello() -> str: + return "Hello, Docker!" + + +@app.post("/heroes/") +def create_hero(hero: Hero) -> Hero: + with Session(engine) as session: + session.add(hero) + session.commit() + session.refresh(hero) + return hero + + +@app.get("/heroes/") +def read_heroes() -> Sequence[Hero]: + with Session(engine) as session: + heroes = session.exec(select(Hero)).all() + return heroes +``` +{{< /file >}} + +{{< file path="config.py" >}} +```python +# Pydantic settings that read PostgreSQL connection details from the +# environment. Supports a password file (Docker secrets) via +# POSTGRES_PASSWORD_FILE in addition to POSTGRES_PASSWORD. +# See https://docs.pydantic.dev/latest/concepts/pydantic_settings/ + +import os +from typing import Any + +from pydantic import ( + PostgresDsn, + computed_field, + field_validator, + model_validator, +) +from pydantic_core import MultiHostUrl +from pydantic_settings import BaseSettings + + +class Settings(BaseSettings): + POSTGRES_SERVER: str + POSTGRES_PORT: int = 5432 + POSTGRES_USER: str + POSTGRES_PASSWORD: str | None = None + POSTGRES_PASSWORD_FILE: str | None = None + POSTGRES_DB: str + + @model_validator(mode="before") + @classmethod + def check_postgres_password(cls, data: Any) -> Any: + """Validate that either POSTGRES_PASSWORD or POSTGRES_PASSWORD_FILE is set.""" + if isinstance(data, dict): + password_file: str | None = data.get("POSTGRES_PASSWORD_FILE") # type: ignore + password: str | None = data.get("POSTGRES_PASSWORD") # type: ignore + if password_file is None and password is None: + raise ValueError( + "At least one of POSTGRES_PASSWORD_FILE and POSTGRES_PASSWORD must be set." + ) + return data # type: ignore + + @field_validator("POSTGRES_PASSWORD_FILE", mode="before") + @classmethod + def read_password_from_file(cls, v: str | None) -> str | None: + if v is not None: + file_path = v + if os.path.exists(file_path): + with open(file_path) as file: + return file.read().strip() + raise ValueError(f"Password file {file_path} does not exist.") + return v + + @computed_field + @property + def SQLALCHEMY_DATABASE_URI(self) -> PostgresDsn: + url = MultiHostUrl.build( + scheme="postgresql+psycopg", + username=self.POSTGRES_USER, + password=self.POSTGRES_PASSWORD + if self.POSTGRES_PASSWORD + else self.POSTGRES_PASSWORD_FILE, + host=self.POSTGRES_SERVER, + port=self.POSTGRES_PORT, + path=self.POSTGRES_DB, + ) + return PostgresDsn(url) + + +settings = Settings() # type: ignore +``` +{{< /file >}} + +{{< file path="requirements.txt" >}} +```text +# Python package dependencies for the application, pinned for reproducible builds. +# See https://pip.pypa.io/en/stable/reference/requirements-file-format/ + +fastapi==0.115.12 +sqlmodel==0.0.24 +psycopg[binary]==3.2.9 +pydantic-settings==2.9.1 +uvicorn==0.34.3 +``` +{{< /file >}} + +{{< file path="Dockerfile" >}} +```dockerfile +# syntax=docker/dockerfile:1 + +# Comments are provided throughout this file to help you get started. +# If you need more help, visit the Dockerfile reference guide at +# https://docs.docker.com/go/dockerfile-reference/ + +# This Dockerfile uses Docker Hardened Images (DHI) for enhanced security. +# For more information, see https://docs.docker.com/dhi/ + +# Use the dev image to build and install dependencies. +# The builder stage is also used directly in development (see compose.yaml). +FROM dhi.io/python:3.12-dev AS builder + +WORKDIR /app + +RUN python3 -m venv /venv +ENV PATH="/venv/bin:$PATH" + +# Download dependencies as a separate step to take advantage of Docker's caching. +# Leverage a cache mount to /root/.cache/pip to speed up subsequent builds. +# Leverage a bind mount to requirements.txt to avoid having to copy them +# into this layer. +RUN --mount=type=cache,target=/root/.cache/pip \ + --mount=type=bind,source=requirements.txt,target=requirements.txt \ + pip install -r requirements.txt + +# Copy the source code into the container. +COPY . . + +# Expose the port that the application listens on. +EXPOSE 8000 + +# Run the application. +CMD ["/venv/bin/python3", "-m", "uvicorn", "app:app", "--host=0.0.0.0", "--port=8000"] + + +# Use the minimal runtime image for production. It runs as nonroot by default. +FROM dhi.io/python:3.12 + +WORKDIR /app + +COPY --from=builder /venv /venv +ENV PATH="/venv/bin:$PATH" + +COPY --from=builder /app . + +EXPOSE 8000 + +CMD ["/venv/bin/python3", "-m", "uvicorn", "app:app", "--host=0.0.0.0", "--port=8000"] +``` +{{< /file >}} + +{{< file path="compose.yaml" status="modified" hl_lines="21-24" >}} +```yaml +services: + # Application service. The `target: builder` line builds the development + # image (includes a shell and tools); the production stage of the + # Dockerfile is unused in development. + server: + build: + context: . + target: builder + ports: + - 8000:8000 + environment: + - POSTGRES_SERVER=db + - POSTGRES_USER=postgres + - POSTGRES_DB=example + - POSTGRES_PASSWORD_FILE=/run/secrets/db-password + depends_on: + db: + condition: service_healthy + secrets: + - db-password + develop: + watch: + - action: rebuild + path: . + db: + image: dhi.io/postgres:18 + restart: always + user: postgres + secrets: + - db-password + volumes: + - db-data:/var/lib/postgresql + environment: + - POSTGRES_DB=example + - POSTGRES_PASSWORD_FILE=/run/secrets/db-password + expose: + - 5432 + healthcheck: + test: ["CMD", "pg_isready"] + interval: 10s + timeout: 5s + retries: 5 +volumes: + db-data: +secrets: + db-password: + file: db/password.txt +``` +{{< /file >}} + +{{< file path="db/password.txt" >}} +```text +mysecretpassword +``` +{{< /file >}} + +{{< file path=".dockerignore" >}} +```text +# Include any files or directories that you don't want to be copied to your +# container here (e.g., local build artifacts, temporary files, etc.). +# +# For more help, visit the .dockerignore file reference guide at +# https://docs.docker.com/go/build-context-dockerignore/ + +**/.DS_Store +**/__pycache__ +**/.venv +**/.classpath +**/.dockerignore +**/.env +**/.git +**/.gitignore +**/.project +**/.settings +**/.toolstarget +**/.vs +**/.vscode +**/*.*proj.user +**/*.dbmdl +**/*.jfm +**/bin +**/charts +**/docker-compose* +**/compose.y*ml +**/Dockerfile* +**/node_modules +**/npm-debug.log +**/obj +**/secrets.dev.yaml +**/values.dev.yaml +LICENSE +README.md +``` +{{< /file >}} + +{{< file path=".gitignore" >}} +```text +# Files and directories that Git should ignore. This is the standard Python +# template covering bytecode, build artifacts, virtual environments, and IDE +# settings. See https://git-scm.com/docs/gitignore for syntax reference. + +# Byte-compiled / optimized / DLL files +__pycache__/ +*.py[cod] +*$py.class + +# C extensions +*.so + +# Distribution / packaging +.Python +build/ +develop-eggs/ +dist/ +downloads/ +eggs/ +.eggs/ +lib/ +lib64/ +parts/ +sdist/ +var/ +wheels/ +share/python-wheels/ +*.egg-info/ +.installed.cfg +*.egg +MANIFEST + +# Unit test / coverage reports +htmlcov/ +.tox/ +.nox/ +.coverage +.coverage.* +.cache +nosetests.xml +coverage.xml +*.cover +*.py,cover +.hypothesis/ +.pytest_cache/ +cover/ + +# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm +__pypackages__/ + +# Environments +.env +.venv +env/ +venv/ +ENV/ +env.bak/ +venv.bak/ + +# Secrets +db/password.txt +``` +{{< /file >}} + +{{< /files >}} + +Run the following command to run your application with Compose Watch. + +```console +$ docker compose watch +``` + +In a terminal, curl the application to get a response. + +```console +$ curl http://localhost:8000 +Hello, Docker! +``` + +Any changes to the application's source files on your local machine will now be immediately reflected in the running container. + +Open `python-docker-example/app.py` in an IDE or text editor and update the `Hello, Docker!` string by adding a few more exclamation marks. + +```diff +- return 'Hello, Docker!' ++ return 'Hello, Docker!!!' +``` + +Save the changes to `app.py` and then wait a few seconds for the application to rebuild. Curl the application again and verify that the updated text appears. + +```console +$ curl http://localhost:8000 +Hello, Docker!!! +``` + +Press `ctrl+c` in the terminal to stop your application. + +### Summary + +In this section, you took a look at setting up your Compose file to add a local +database and persist data. You also learned how to use Compose Watch to automatically rebuild and run your container when you update your code. + +Related information: + +- [Compose file reference](/reference/compose-file/) +- [Compose secrets](/reference/compose-file/secrets.md) +- [Compose Watch](/manuals/compose/how-tos/file-watch.md) +- [Multi-stage builds](/manuals/build/building/multi-stage.md) + +### Next steps + +In the next section, you'll learn how you can set up linting, formatting, and type checking to follow the best practices in Python apps. + +## Linting, formatting, and type checking for Python + +### Prerequisites + +Complete [Develop your app](develop.md). This topic requires a local Python +installation because the tools and Git hooks introduced here run on your +host. If you don't want to install Python locally, skip this topic. The same +checks run in CI in the [next topic](configure-github-actions.md). + +### Overview + +Linting, formatting, and type checking are automated ways to catch bugs, +enforce style, and spot type errors before code runs. Running them on every +commit, in CI, and in your editor catches problems early when they're cheap +to fix. + +In this section, you'll configure three tools for your Python application. +Ruff handles linting and formatting in a single fast pass. Pyright statically +checks your code for type errors. Pre-commit hooks run both of these +automatically before each Git commit so problems are caught locally before +they're committed. + +### Linting and formatting with Ruff + +Ruff is an extremely fast Python linter and formatter written in Rust. It replaces multiple tools like flake8, isort, and black with a single unified tool. + +Create a `pyproject.toml` file in your `python-docker-example` directory: + +{{< files name="python-docker-example" >}} + +{{< file path="pyproject.toml" status="new" >}} +```toml +# Configuration for code-quality tools. +# - [tool.ruff]: linting and formatting (https://docs.astral.sh/ruff/) +# - [tool.pyright]: static type checking (https://microsoft.github.io/pyright/) + +[tool.ruff] +target-version = "py312" + +[tool.ruff.lint] +select = [ + "E", # pycodestyle errors + "W", # pycodestyle warnings + "F", # pyflakes + "I", # isort + "B", # flake8-bugbear + "C4", # flake8-comprehensions + "UP", # pyupgrade + "ARG001", # unused arguments in functions +] +ignore = [ + "E501", # line too long, handled by black + "B008", # do not perform function calls in argument defaults + "W191", # indentation contains tabs + "B904", # Allow raising exceptions without from e, for HTTPException +] +``` +{{< /file >}} + +{{< /files >}} + +Install Ruff: + +```console +$ pip install ruff +``` + +If you're using a virtual environment, make sure it is activated so the `ruff` +command is available. + +Run these commands to check and format your code: + +```console +# Check for errors +$ ruff check . + +# Automatically fix fixable errors +$ ruff check --fix . + +# Format code +$ ruff format . +``` + +### Type checking with Pyright + +Pyright is a fast static type checker for Python that works well with modern Python features. + +Update `pyproject.toml` to add the Pyright configuration at the bottom. + +{{< files name="python-docker-example" >}} + +{{< file path="pyproject.toml" status="modified" hl_lines="25-29" >}} +```toml +# Configuration for code-quality tools. +# - [tool.ruff]: linting and formatting (https://docs.astral.sh/ruff/) +# - [tool.pyright]: static type checking (https://microsoft.github.io/pyright/) + +[tool.ruff] +target-version = "py312" + +[tool.ruff.lint] +select = [ + "E", # pycodestyle errors + "W", # pycodestyle warnings + "F", # pyflakes + "I", # isort + "B", # flake8-bugbear + "C4", # flake8-comprehensions + "UP", # pyupgrade + "ARG001", # unused arguments in functions +] +ignore = [ + "E501", # line too long, handled by black + "B008", # do not perform function calls in argument defaults + "W191", # indentation contains tabs + "B904", # Allow raising exceptions without from e, for HTTPException +] + +[tool.pyright] +typeCheckingMode = "strict" +pythonVersion = "3.12" +exclude = [".venv"] +``` +{{< /file >}} + +{{< /files >}} + +Install Pyright and run it: + +```console +$ pip install pyright +$ pyright +``` + +### Setting up pre-commit hooks + +Pre-commit hooks run checks automatically before each commit on your local +machine. Create a `.pre-commit-config.yaml` file in your `python-docker-example` +directory to set up Ruff hooks: + +{{< files name="python-docker-example" >}} + +{{< file path=".pre-commit-config.yaml" status="new" >}} +```yaml +# Pre-commit hook configuration. Runs Ruff (lint + format) on every +# `git commit`. See https://pre-commit.com/ + +repos: + - repo: https://github.com/astral-sh/ruff-pre-commit + rev: v0.15.15 + hooks: + - id: ruff + args: [--fix] + - id: ruff-format +``` +{{< /file >}} + +{{< /files >}} + +To install and use: + +```console +$ pip install pre-commit +$ pre-commit install +$ git commit -m "Test commit" # Automatically runs checks +``` + +### Summary + +In this section, you learned how to: + +- Configure and use Ruff for linting and formatting +- Set up Pyright for static type checking +- Automate checks with pre-commit hooks + +These tools help maintain code quality and catch errors early in development. + +Related information: + +- [Ruff documentation](https://docs.astral.sh/ruff/) +- [Pyright documentation](https://microsoft.github.io/pyright/) +- [pre-commit framework](https://pre-commit.com/) + +### Next steps + +- [Configure GitHub Actions](configure-github-actions.md) to run these checks automatically +- Customize linting rules to match your team's style preferences +- Explore advanced type checking features + +## Automate your builds with GitHub Actions + +### Prerequisites + +Complete all the previous sections of this guide, starting with [Containerize a Python application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. + +If you didn't create a [GitHub repository](https://github.com/new) for your project yet, it is time to do it. After creating the repository, don't forget to [add a remote](https://docs.github.com/en/get-started/getting-started-with-git/managing-remote-repositories) and ensure you can commit and [push your code](https://docs.github.com/en/get-started/using-git/pushing-commits-to-a-remote-repository#about-git-push) to GitHub. + +1. In your project's GitHub repository, open **Settings**, and go to **Secrets and variables** > **Actions**. + +2. Under the **Variables** tab, create a new **Repository variable** named `DOCKER_USERNAME` and your Docker ID as a value. + +3. Create a new [Personal Access Token (PAT)](/manuals/security/access-tokens.md#create-an-access-token) for Docker Hub. You can name this token `docker-tutorial`. Make sure access permissions include Read and Write. + +4. Add the PAT as a **Repository secret** in your GitHub repository, with the name + `DOCKERHUB_TOKEN`. + +### Overview + +GitHub Actions is a CI/CD automation tool built into GitHub. A workflow is a +YAML file that tells GitHub which jobs to run when something happens in your +repository, like a push to a branch or a pull request opening. Workflows live +in the `.github/workflows/` directory of your repository. + +In this section, you'll add a workflow that runs your linting, formatting, and +type checks on every push to the main branch, then builds your Docker image +and pushes it to Docker Hub. + +### 1. Define the GitHub Actions workflow + +You can create a GitHub Actions workflow by creating a YAML file in the `.github/workflows/` directory of your repository. To do this use your favorite text editor or the GitHub web interface. The following steps show you how to create a workflow file using the GitHub web interface. + +If you prefer to use the GitHub web interface, follow these steps: + +1. Go to your repository on GitHub and then select the **Actions** tab. + +2. Select **set up a workflow yourself**. + + This takes you to a page for creating a new GitHub Actions workflow file in + your repository. By default, the file is created under `.github/workflows/main.yml`. Change the file name to `build.yml`. + +If you prefer to use your text editor, create a new file named `build.yml` in the `.github/workflows/` directory of your repository. + +Add the following content to the file: + +{{< files name="python-docker-example" >}} + +{{< file path=".github/workflows/build.yml" status="new" >}} +```yaml +# GitHub Actions workflow that runs on every push to main. +# - lint-test: runs pre-commit hooks (Ruff) and Pyright type checks. +# - build_and_push: signs in to Docker Hub and the DHI registry, then +# builds and pushes the image (with SBOM and provenance attestations). +name: Build and push Docker image + +on: + push: + branches: + - main + +jobs: + lint-test: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@{{% param "checkout_action_version" %}} + + - name: Set up Python + uses: actions/setup-python@v6 + with: + python-version: '3.12' + + - name: Install dependencies + run: | + python -m pip install --upgrade pip + pip install -r requirements.txt + pip install pre-commit pyright + + - name: Run pre-commit hooks + run: pre-commit run --all-files + + - name: Run pyright + run: pyright + + build_and_push: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@{{% param "checkout_action_version" %}} + + - name: Login to Docker Hub + uses: docker/login-action@{{% param "login_action_version" %}} + with: + username: ${{ vars.DOCKER_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + + - name: Login to Docker Hardened Images + uses: docker/login-action@{{% param "login_action_version" %}} + with: + registry: dhi.io + username: ${{ vars.DOCKER_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} + + - name: Build and push + uses: docker/build-push-action@{{% param "build_push_action_version" %}} + with: + push: true + tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest +``` +{{< /file >}} + +{{< /files >}} + +Each GitHub Actions workflow includes one or several jobs. Each job consists of steps. Each step can either run a set of commands or use already [existing actions](https://github.com/marketplace?type=actions). The action above has three steps: + +1. [Login to Docker Hub](https://github.com/docker/login-action): Action logs in to Docker Hub using the Docker ID and Personal Access Token (PAT) you created earlier. + +2. [Set up Docker Buildx](https://github.com/docker/setup-buildx-action): Action sets up Docker [Buildx](https://github.com/docker/buildx), a CLI plugin that extends the capabilities of the Docker CLI. + +3. [Build and push](https://github.com/docker/build-push-action): Action builds and pushes the Docker image to Docker Hub. The `tags` parameter specifies the image name and tag. The `latest` tag is used in this example. + +### 2. Run the workflow + +Commit the changes and push them to the `main` branch. This workflow is runs every time you push changes to the `main` branch. You can find more information about workflow triggers [in the GitHub documentation](https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows). + +Go to the **Actions** tab of you GitHub repository. It displays the workflow. Selecting the workflow shows you the breakdown of all the steps. + +When the workflow is complete, go to your [repositories on Docker Hub](https://hub.docker.com/repositories). If you see the new repository in that list, it means the GitHub Actions workflow successfully pushed the image to Docker Hub. + +### Summary + +In this section, you learned how to set up a GitHub Actions workflow for your Python application that includes: + +- Running pre-commit hooks for linting and formatting +- Static type checking with Pyright +- Building and pushing Docker images + +Related information: + +- [Introduction to GitHub Actions](/guides/gha.md) +- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) +- [docker/login-action](https://github.com/docker/login-action) +- [docker/build-push-action](https://github.com/docker/build-push-action) +- [Create a Docker Hub access token](/manuals/security/access-tokens.md#create-an-access-token) + +### Next steps + +In the next section, you'll learn how to inspect and generate supply chain +attestations for your image. See [Secure your supply chain](secure-supply-chain.md). + +## Secure your Python image supply chain + +### Prerequisites + +Complete [Configure CI/CD for your Python application](configure-github-actions.md). + +### Overview + +When you ship a container image, what's inside it and where it came from +matters. Supply chain attestations are signed records that answer questions +like which packages are in the image, what vulnerabilities affect them, how +the image was built, and what security checks it passed. + +In this section, you'll inspect the attestations that ship with your Docker +Hardened Image base, generate your own SBOM and provenance attestations +during CI, and pin the base image by digest so your builds are reproducible. + +The inspection commands in this topic are shown manually so you can see what +each one returns. In a real workflow you'd automate these checks with +[Docker Scout](/scout/), which runs the same scans on every push, +enforces policies in CI, and surfaces results in your registry and pull +requests. + +### Inspect the base image attestations + +Docker Hardened Images are built to SLSA Build Level 3 and ship with a set of +signed attestations covering bill-of-materials, vulnerabilities, build +provenance, and security scans. See +[DHI attestations](/manuals/dhi/core-concepts/attestations.md) for the full +list of types and how to verify their signatures with Cosign. + +List all the attestations available on the Python DHI: + +```console +$ docker scout attest list registry://dhi.io/python:3.12 +``` + +View the SBOM: + +```console +$ docker scout sbom registry://dhi.io/python:3.12 +``` + +Check known vulnerabilities: + +```console +$ docker scout cves registry://dhi.io/python:3.12 +``` + +> [!NOTE] +> +> The `registry://` prefix forces `docker scout` to fetch the image and its +> attestations from the registry instead of reading a locally pulled copy. If +> you've already pulled or built against the base image, the local copy +> doesn't have the attached attestations, so the prefix is required to see +> them. + +When you base your own image on a DHI image, these attestations stay attached to the base layer in the registry. Tools that inspect your image can follow the chain back to the DHI source. + +### Generate attestations for your image + +Update your GitHub Actions workflow to attach SBOM and provenance attestations to the image you push. + +Edit `.github/workflows/build.yml` and update the build-and-push step: + +```yaml {hl_lines="6-7"} +- name: Build and push Docker image + uses: docker/build-push-action@v6 + with: + context: . + push: true + sbom: true + provenance: mode=max + tags: ${{ steps.meta.outputs.tags }} +``` + +- `sbom: true` tells BuildKit to scan the built image and attach an SBOM attestation. +- `provenance: mode=max` records detailed build provenance, including the source repository, commit, and build parameters. + +The next time your workflow runs, the pushed image will carry these attestations alongside the image manifest in the registry. + +### Inspect your pushed image's attestations + +After your workflow pushes the image, inspect it the same way you inspected the base image: + +```console +$ docker scout attest list registry://DOCKER_USERNAME/REPO_NAME:latest +$ docker scout sbom registry://DOCKER_USERNAME/REPO_NAME:latest +``` + +The SBOM includes packages from every layer, including those inherited from `dhi.io/python:3.12`. The provenance record references the DHI base image by digest, so consumers of your image can trace the build chain back to the DHI source. + +### Pin the base image by digest + +Image tags like `dhi.io/python:3.12` move over time as new patches land. For reproducible builds, pin to an immutable digest. + +The Dockerfile uses two tags, `dhi.io/python:3.12-dev` in the builder stage +and `dhi.io/python:3.12` in the runtime stage. Each tag has its own digest, +so look up both: + +```console +$ docker buildx imagetools inspect dhi.io/python:3.12-dev --format "{{ .Manifest.Digest }}" +sha256:4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 +$ docker buildx imagetools inspect dhi.io/python:3.12 --format "{{ .Manifest.Digest }}" +sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 +``` + +Each digest is a 64-character hex string. Update your `Dockerfile` to reference +each digest on the matching `FROM` line: + +```dockerfile +FROM dhi.io/python:3.12-dev@sha256:4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 AS builder +# ... +FROM dhi.io/python:3.12@sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 +``` + +> [!TIP] +> +> Pinning by digest also pins you to that image's vulnerabilities. Use [Dependabot](https://docs.github.com/en/code-security/dependabot) or [Renovate](https://docs.renovatebot.com/) to automate digest updates so you get a PR when a new patched image is available, with a changelog to review before merging. + +### Summary + +In this section, you learned how to: + +- Inspect the supply chain attestations that ship with the DHI base image, including SBOMs, CVE reports, VEX statements, and scan results +- Generate SBOM and provenance attestations for your own image in CI +- Pin base images by digest for reproducible builds + +Related information: + +- [DHI attestations](/manuals/dhi/core-concepts/attestations.md) +- [Verify a Docker Hardened Image](/manuals/dhi/how-to/verify.md) +- [Docker Scout](/scout/) +- [Build attestations](/manuals/build/metadata/attestations/_index.md) + +### Next steps + +In the next section, you'll deploy your application to Kubernetes. + +## Test your Python deployment + +### Prerequisites + +- Complete all the previous sections of this guide, starting with [Use containers for Python development](develop.md). +- [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. + +### Overview + +[Kubernetes](https://kubernetes.io/) is an open source platform that runs and +orchestrates container workloads across one or more machines. You describe +what you want to run, like which container images, how many replicas, and +which network ports to expose, in YAML manifest files. Kubernetes reads the +manifests and makes the cluster match that description. + +In this section, you'll use the Kubernetes environment built into Docker +Desktop to deploy your application locally. You'll write two manifest files, +one for the PostgreSQL database and one for the FastAPI application, apply +them with `kubectl`, and verify the deployment by hitting your application +from a terminal. + +### Registry authentication + +The Docker Hardened Images used in this guide are hosted on `dhi.io`. Docker +Desktop's Kubernetes shares credentials with Docker Desktop, so the `docker login dhi.io` +you completed earlier is all that's needed. No additional image pull secret is required. + +> [!NOTE] +> +> If you're deploying to a Kubernetes cluster outside of Docker Desktop, you'll +> need to create an image pull secret and reference it in your pod specs. See +> [Use a Docker Hardened Image](/dhi/how-to/use/#use-with-kubernetes) for instructions. + +### Create a Kubernetes YAML file + +Create the following two Kubernetes manifest files in your +`python-docker-example` directory. Before applying +`docker-python-kubernetes.yaml`, replace `DOCKER_USERNAME/REPO_NAME` with your +Docker username and the repository name that you created in [Configure CI/CD for +your Python application](./configure-github-actions.md). + +{{< files name="python-docker-example" >}} + +{{< file path="docker-postgres-kubernetes.yaml" status="new" >}} +```yaml +# Kubernetes manifests for the PostgreSQL database used by the FastAPI app. +# Contains a Deployment, Service, PersistentVolumeClaim, and Secret. + +# Deployment: runs one PostgreSQL pod. The image, port, env vars, and the +# persistent volume mount are all defined here. +apiVersion: apps/v1 +kind: Deployment +metadata: + name: postgres + namespace: default +spec: + replicas: 1 + selector: + matchLabels: + app: postgres + template: + metadata: + labels: + app: postgres + spec: + containers: + - name: postgres + image: dhi.io/postgres:18 + ports: + - containerPort: 5432 + env: + - name: POSTGRES_DB + value: example + - name: POSTGRES_USER + value: postgres + - name: POSTGRES_PASSWORD + valueFrom: + secretKeyRef: + name: postgres-secret + key: POSTGRES_PASSWORD + volumeMounts: + - name: postgres-data + mountPath: /var/lib/postgresql + volumes: + - name: postgres-data + persistentVolumeClaim: + claimName: postgres-pvc +--- +# Service: exposes PostgreSQL inside the cluster on port 5432 so the +# application pod can reach it by the DNS name `postgres`. +apiVersion: v1 +kind: Service +metadata: + name: postgres + namespace: default +spec: + ports: + - port: 5432 + selector: + app: postgres +--- +# PersistentVolumeClaim: storage that survives pod restarts. +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: postgres-pvc + namespace: default +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi +--- +# Secret: holds the database password (base64-encoded). Referenced by both +# the postgres Deployment and the application Deployment. +apiVersion: v1 +kind: Secret +metadata: + name: postgres-secret + namespace: default +type: Opaque +data: + POSTGRES_PASSWORD: cG9zdGdyZXNfcGFzc3dvcmQ= # Base64 encoded password (e.g., 'postgres_password') +``` +{{< /file >}} + +{{< file path="docker-python-kubernetes.yaml" status="new" >}} +```yaml +# Kubernetes manifests for the FastAPI application. +# Contains a Deployment and a NodePort Service. + +# Deployment: runs the FastAPI app. Connection details to the postgres +# service are passed in via environment variables, and the database +# password comes from the shared postgres-secret. +apiVersion: apps/v1 +kind: Deployment +metadata: + name: docker-python-demo + namespace: default +spec: + replicas: 1 + selector: + matchLabels: + service: fastapi + template: + metadata: + labels: + service: fastapi + spec: + containers: + - name: fastapi-service + image: DOCKER_USERNAME/REPO_NAME + imagePullPolicy: Always + env: + - name: POSTGRES_PASSWORD + valueFrom: + secretKeyRef: + name: postgres-secret + key: POSTGRES_PASSWORD + - name: POSTGRES_USER + value: postgres + - name: POSTGRES_DB + value: example + - name: POSTGRES_SERVER + value: postgres + - name: POSTGRES_PORT + value: "5432" + ports: + - containerPort: 8000 +--- +# Service: exposes the FastAPI app on port 30001 of the cluster node so +# you can reach it from your host with `curl http://localhost:30001/`. +apiVersion: v1 +kind: Service +metadata: + name: service-entrypoint + namespace: default +spec: + type: NodePort + selector: + service: fastapi + ports: + - port: 8000 + targetPort: 8000 + nodePort: 30001 +``` +{{< /file >}} + +{{< /files >}} + +In these Kubernetes YAML files, there are various objects, separated by the `---`: + +- A Deployment, describing a scalable group of identical pods. In this case, + you'll get just one replica, or copy of your pod. That pod, which is + described under `template`, has just one container in it. The + container is created from the image built by GitHub Actions in [Configure CI/CD for + your Python application](configure-github-actions.md). +- A Service, which will define how the ports are mapped in the containers. +- A PersistentVolumeClaim, to define a storage that will be persistent through restarts for the database. +- A Secret, which stores the database password as a Kubernetes Secret resource. +- A NodePort service, which will route traffic from port 30001 on your host to + port 8000 inside the pods it routes to, so you can reach your app + from the network. + +To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). + +> [!NOTE] +> +> The `NodePort` service is good for development and testing. For production, implement an [ingress controller](https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/) instead. + +### Deploy and check your application + +1. In a terminal, navigate to `python-docker-example` and deploy your database to + Kubernetes. + + ```console + $ kubectl apply -f docker-postgres-kubernetes.yaml + ``` + + You should see output that looks like the following, indicating your Kubernetes objects were created successfully. + + ```console + deployment.apps/postgres created + service/postgres created + persistentvolumeclaim/postgres-pvc created + secret/postgres-secret created + ``` + + Now, deploy your Python application. + + ```console + $ kubectl apply -f docker-python-kubernetes.yaml + ``` + + You should see output that looks like the following, indicating your Kubernetes objects were created successfully. + + ```console + deployment.apps/docker-python-demo created + service/service-entrypoint created + ``` + +2. Make sure everything worked by listing your deployments. + + ```console + $ kubectl get deployments + ``` + + Your deployment should be listed as follows: + + ```console + NAME READY UP-TO-DATE AVAILABLE AGE + docker-python-demo 1/1 1 1 48s + postgres 1/1 1 1 2m39s + ``` + + This indicates all one of the pods you asked for in your YAML are up and running. Do the same check for your services. + + ```console + $ kubectl get services + ``` + + You should get output like the following. + + ```console + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + kubernetes ClusterIP 10.43.0.1 443/TCP 13h + postgres ClusterIP 10.43.209.25 5432/TCP 3m10s + service-entrypoint NodePort 10.43.67.120 8000:30001/TCP 79s + ``` + + In addition to the default `kubernetes` service, you can see your `service-entrypoint` service, accepting traffic on port 30001/TCP and the internal `ClusterIP` `postgres` with the port `5432` open to accept connections from your Python app. + +3. In a terminal, curl the root endpoint to verify the application is running. + + ```console + $ curl http://localhost:30001/ + Hello, Docker! + ``` + +4. Exercise the database by creating a hero with a POST request: + + ```console + $ curl -X 'POST' \ + 'http://localhost:30001/heroes/' \ + -H 'accept: application/json' \ + -H 'Content-Type: application/json' \ + -d '{ + "id": 1, + "name": "my hero", + "secret_name": "austing", + "age": 12 + }' + ``` + + You should receive the following response: + + ```json + { + "age": 12, + "id": 1, + "name": "my hero", + "secret_name": "austing" + } + ``` + + Then read it back with a GET request: + + ```console + $ curl http://localhost:30001/heroes/ + ``` + + You should receive an array containing the hero you just created. This + confirms the application can read from and write to the PostgreSQL database + running in the cluster. + +5. Run the following commands to tear down your application. + + ```console + $ kubectl delete -f docker-python-kubernetes.yaml + $ kubectl delete -f docker-postgres-kubernetes.yaml + ``` + +### Summary + +In this section, you learned how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. + +Related information: + +- [Kubernetes documentation](https://kubernetes.io/docs/home/) +- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) +- [Use a Docker Hardened Image with Kubernetes](/dhi/how-to/use/#use-with-kubernetes) diff --git a/content/guides/python/configure-github-actions.md b/content/guides/python/configure-github-actions.md deleted file mode 100644 index e935147bec1a..000000000000 --- a/content/guides/python/configure-github-actions.md +++ /dev/null @@ -1,161 +0,0 @@ ---- -title: Automate your builds with GitHub Actions -linkTitle: GitHub Actions CI -weight: 40 -keywords: ci/cd, github actions, python, flask -description: Learn how to configure CI/CD using GitHub Actions for your Python application. -aliases: - - /language/python/configure-ci-cd/ - - /guides/language/python/configure-ci-cd/ - - /guides/python/configure-ci-cd/ ---- - -## Prerequisites - -Complete all the previous sections of this guide, starting with [Containerize a Python application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. - -If you didn't create a [GitHub repository](https://github.com/new) for your project yet, it is time to do it. After creating the repository, don't forget to [add a remote](https://docs.github.com/en/get-started/getting-started-with-git/managing-remote-repositories) and ensure you can commit and [push your code](https://docs.github.com/en/get-started/using-git/pushing-commits-to-a-remote-repository#about-git-push) to GitHub. - -1. In your project's GitHub repository, open **Settings**, and go to **Secrets and variables** > **Actions**. - -2. Under the **Variables** tab, create a new **Repository variable** named `DOCKER_USERNAME` and your Docker ID as a value. - -3. Create a new [Personal Access Token (PAT)](/manuals/security/access-tokens.md#create-an-access-token) for Docker Hub. You can name this token `docker-tutorial`. Make sure access permissions include Read and Write. - -4. Add the PAT as a **Repository secret** in your GitHub repository, with the name - `DOCKERHUB_TOKEN`. - -## Overview - -GitHub Actions is a CI/CD automation tool built into GitHub. A workflow is a -YAML file that tells GitHub which jobs to run when something happens in your -repository, like a push to a branch or a pull request opening. Workflows live -in the `.github/workflows/` directory of your repository. - -In this section, you'll add a workflow that runs your linting, formatting, and -type checks on every push to the main branch, then builds your Docker image -and pushes it to Docker Hub. - -## 1. Define the GitHub Actions workflow - -You can create a GitHub Actions workflow by creating a YAML file in the `.github/workflows/` directory of your repository. To do this use your favorite text editor or the GitHub web interface. The following steps show you how to create a workflow file using the GitHub web interface. - -If you prefer to use the GitHub web interface, follow these steps: - -1. Go to your repository on GitHub and then select the **Actions** tab. - -2. Select **set up a workflow yourself**. - - This takes you to a page for creating a new GitHub Actions workflow file in - your repository. By default, the file is created under `.github/workflows/main.yml`. Change the file name to `build.yml`. - -If you prefer to use your text editor, create a new file named `build.yml` in the `.github/workflows/` directory of your repository. - -Add the following content to the file: - -{{< files name="python-docker-example" >}} - -{{< file path=".github/workflows/build.yml" status="new" >}} -```yaml -# GitHub Actions workflow that runs on every push to main. -# - lint-test: runs pre-commit hooks (Ruff) and Pyright type checks. -# - build_and_push: signs in to Docker Hub and the DHI registry, then -# builds and pushes the image (with SBOM and provenance attestations). -name: Build and push Docker image - -on: - push: - branches: - - main - -jobs: - lint-test: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@{{% param "checkout_action_version" %}} - - - name: Set up Python - uses: actions/setup-python@v6 - with: - python-version: '3.12' - - - name: Install dependencies - run: | - python -m pip install --upgrade pip - pip install -r requirements.txt - pip install pre-commit pyright - - - name: Run pre-commit hooks - run: pre-commit run --all-files - - - name: Run pyright - run: pyright - - build_and_push: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@{{% param "checkout_action_version" %}} - - - name: Login to Docker Hub - uses: docker/login-action@{{% param "login_action_version" %}} - with: - username: ${{ vars.DOCKER_USERNAME }} - password: ${{ secrets.DOCKERHUB_TOKEN }} - - - name: Login to Docker Hardened Images - uses: docker/login-action@{{% param "login_action_version" %}} - with: - registry: dhi.io - username: ${{ vars.DOCKER_USERNAME }} - password: ${{ secrets.DOCKERHUB_TOKEN }} - - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} - - - name: Build and push - uses: docker/build-push-action@{{% param "build_push_action_version" %}} - with: - push: true - tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest -``` -{{< /file >}} - -{{< /files >}} - -Each GitHub Actions workflow includes one or several jobs. Each job consists of steps. Each step can either run a set of commands or use already [existing actions](https://github.com/marketplace?type=actions). The action above has three steps: - -1. [Login to Docker Hub](https://github.com/docker/login-action): Action logs in to Docker Hub using the Docker ID and Personal Access Token (PAT) you created earlier. - -2. [Set up Docker Buildx](https://github.com/docker/setup-buildx-action): Action sets up Docker [Buildx](https://github.com/docker/buildx), a CLI plugin that extends the capabilities of the Docker CLI. - -3. [Build and push](https://github.com/docker/build-push-action): Action builds and pushes the Docker image to Docker Hub. The `tags` parameter specifies the image name and tag. The `latest` tag is used in this example. - -## 2. Run the workflow - -Commit the changes and push them to the `main` branch. This workflow is runs every time you push changes to the `main` branch. You can find more information about workflow triggers [in the GitHub documentation](https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows). - -Go to the **Actions** tab of you GitHub repository. It displays the workflow. Selecting the workflow shows you the breakdown of all the steps. - -When the workflow is complete, go to your [repositories on Docker Hub](https://hub.docker.com/repositories). If you see the new repository in that list, it means the GitHub Actions workflow successfully pushed the image to Docker Hub. - -## Summary - -In this section, you learned how to set up a GitHub Actions workflow for your Python application that includes: - -- Running pre-commit hooks for linting and formatting -- Static type checking with Pyright -- Building and pushing Docker images - -Related information: - -- [Introduction to GitHub Actions](/guides/gha.md) -- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) -- [docker/login-action](https://github.com/docker/login-action) -- [docker/build-push-action](https://github.com/docker/build-push-action) -- [Create a Docker Hub access token](/manuals/security/access-tokens.md#create-an-access-token) - -## Next steps - -In the next section, you'll learn how to inspect and generate supply chain -attestations for your image. See [Secure your supply chain](secure-supply-chain.md). - diff --git a/content/guides/python/containerize.md b/content/guides/python/containerize.md deleted file mode 100644 index 193d191d4f0e..000000000000 --- a/content/guides/python/containerize.md +++ /dev/null @@ -1,439 +0,0 @@ ---- -title: Containerize a Python application -linkTitle: Containerize your app -weight: 10 -keywords: python, flask, containerize, initialize -description: Learn how to containerize a Python application. -aliases: - - /language/python/build-images/ - - /language/python/run-containers/ - - /language/python/containerize/ - - /guides/language/python/containerize/ ---- - -## Prerequisites - -- You have installed the latest version of [Docker Desktop](/get-started/get-docker.md). - -## Overview - -Containerizing your application means packaging it together with its -dependencies, configuration, and runtime into a single portable unit called a -container image. Running that image creates a container, an isolated process -that behaves the same on any machine, whether it's your laptop, a CI runner, or -a production server. - -In this section, you'll containerize a simple -[FastAPI](https://fastapi.tiangolo.com) web application. You'll write a -`Dockerfile` that describes how to build the image, add a `compose.yaml` file -that defines how Docker runs your container, and then build and start the -application with one command. - -You'll use [Docker Hardened Images](/dhi/) as the base. These are minimal, -secure Python images maintained by Docker. - -## Create the application - -The sample application is a minimal FastAPI service with a single endpoint -that returns a JSON greeting. Create the following files in a new -`python-docker-example` directory. To create all the files at once, switch to -the **Scaffold script** tab in the file browser and copy the shell command. - -{{< files name="python-docker-example" >}} - -{{< file path="app.py" status="new" >}} -```python -# A minimal FastAPI application. -# The root endpoint (GET /) returns a JSON "Hello World" response. -# See https://fastapi.tiangolo.com/ for the framework reference. - -from fastapi import FastAPI - -app = FastAPI() - - -@app.get("/") -async def root(): - return {"message": "Hello World"} -``` -{{< /file >}} - -{{< file path="requirements.txt" status="new" >}} -```text -# Python package dependencies for the application, pinned for reproducible builds. -# See https://pip.pypa.io/en/stable/reference/requirements-file-format/ - -fastapi==0.115.12 -uvicorn==0.34.3 -``` -{{< /file >}} - -{{< file path=".gitignore" status="new" >}} -```text -# Files and directories that Git should ignore. This is the standard Python -# template covering bytecode, build artifacts, virtual environments, and IDE -# settings. See https://git-scm.com/docs/gitignore for syntax reference. - -# Byte-compiled / optimized / DLL files -__pycache__/ -*.py[cod] -*$py.class - -# C extensions -*.so - -# Distribution / packaging -.Python -build/ -develop-eggs/ -dist/ -downloads/ -eggs/ -.eggs/ -lib/ -lib64/ -parts/ -sdist/ -var/ -wheels/ -share/python-wheels/ -*.egg-info/ -.installed.cfg -*.egg -MANIFEST - -# Unit test / coverage reports -htmlcov/ -.tox/ -.nox/ -.coverage -.coverage.* -.cache -nosetests.xml -coverage.xml -*.cover -*.py,cover -.hypothesis/ -.pytest_cache/ -cover/ - -# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm -__pypackages__/ - -# Environments -.env -.venv -env/ -venv/ -ENV/ -env.bak/ -venv.bak/ - -# Secrets -db/password.txt -``` -{{< /file >}} - -{{< /files >}} - -If you already have Python installed and want to verify the app works before -containerizing it, you can run it locally: - -```console -$ python3 -m venv .venv -$ source .venv/bin/activate -$ pip install -r requirements.txt -$ uvicorn app:app --reload -``` - -> [!NOTE] -> -> On Windows, activate the virtual environment with `.venv\Scripts\activate` -> instead of `source .venv/bin/activate`. - -If you don't have Python installed, skip ahead to the next section. The -remaining steps run the application in a container, with no local Python -required. - -## Create the Docker assets - -Sign in to the DHI registry so Docker can pull the Python base images during -the build. The available Python images are listed in the -[catalog](https://hub.docker.com/hardened-images/catalog/dhi/python). - -```console -$ docker login dhi.io -``` - -Add the following three files to your `python-docker-example` directory. The -`Dockerfile` describes how to build the image, `compose.yaml` defines how -Docker runs the container, and `.dockerignore` keeps unwanted files out of the -build context. - -> [!TIP] -> -> [Gordon](/ai/gordon/), Docker's AI assistant, can generate Docker assets for -> your project. Ask Gordon to create a Dockerfile, Compose file, and -> `.dockerignore` tailored to your application. - -{{< files name="python-docker-example" >}} - -{{< file path="app.py" >}} -```python -# A minimal FastAPI application. -# The root endpoint (GET /) returns a JSON "Hello World" response. -# See https://fastapi.tiangolo.com/ for the framework reference. - -from fastapi import FastAPI - -app = FastAPI() - - -@app.get("/") -async def root(): - return {"message": "Hello World"} -``` -{{< /file >}} - -{{< file path="requirements.txt" >}} -```text -# Python package dependencies for the application, pinned for reproducible builds. -# See https://pip.pypa.io/en/stable/reference/requirements-file-format/ - -fastapi==0.115.12 -uvicorn==0.34.3 -``` -{{< /file >}} - -{{< file path="Dockerfile" status="new" >}} -```dockerfile -# syntax=docker/dockerfile:1 - -# Comments are provided throughout this file to help you get started. -# If you need more help, visit the Dockerfile reference guide at -# https://docs.docker.com/go/dockerfile-reference/ - -# This Dockerfile uses Docker Hardened Images (DHI) for enhanced security. -# For more information, see https://docs.docker.com/dhi/ - -# Use the dev image to build and install dependencies. -FROM dhi.io/python:3.12-dev AS builder - -WORKDIR /app - -RUN python3 -m venv /venv -ENV PATH="/venv/bin:$PATH" - -# Download dependencies as a separate step to take advantage of Docker's caching. -# Leverage a cache mount to /root/.cache/pip to speed up subsequent builds. -# Leverage a bind mount to requirements.txt to avoid having to copy them into -# this layer. -RUN --mount=type=cache,target=/root/.cache/pip \ - --mount=type=bind,source=requirements.txt,target=requirements.txt \ - pip install -r requirements.txt - -# Use the minimal runtime image. It runs as nonroot by default. -FROM dhi.io/python:3.12 - -WORKDIR /app - -COPY --from=builder /venv /venv -ENV PATH="/venv/bin:$PATH" - -# Copy the source code into the container. -COPY . . - -# Expose the port that the application listens on. -EXPOSE 8000 - -# Run the application. -CMD ["/venv/bin/python3", "-m", "uvicorn", "app:app", "--host=0.0.0.0", "--port=8000"] -``` -{{< /file >}} - -{{< file path="compose.yaml" status="new" >}} -```yaml -# Comments are provided throughout this file to help you get started. -# If you need more help, visit the Docker Compose reference guide at -# https://docs.docker.com/go/compose-spec-reference/ - -# Here the instructions define your application as a service called "server". -# This service is built from the Dockerfile in the current directory. -# You can add other services your application may depend on here, such as a -# database or a cache. For examples, see the Awesome Compose repository: -# https://github.com/docker/awesome-compose -services: - server: - build: - context: . - ports: - - 8000:8000 -``` -{{< /file >}} - -{{< file path=".dockerignore" status="new" >}} -```text -# Include any files or directories that you don't want to be copied to your -# container here (e.g., local build artifacts, temporary files, etc.). -# -# For more help, visit the .dockerignore file reference guide at -# https://docs.docker.com/go/build-context-dockerignore/ - -**/.DS_Store -**/__pycache__ -**/.venv -**/.classpath -**/.dockerignore -**/.env -**/.git -**/.gitignore -**/.project -**/.settings -**/.toolstarget -**/.vs -**/.vscode -**/*.*proj.user -**/*.dbmdl -**/*.jfm -**/bin -**/charts -**/docker-compose* -**/compose.y*ml -**/Dockerfile* -**/node_modules -**/npm-debug.log -**/obj -**/secrets.dev.yaml -**/values.dev.yaml -LICENSE -README.md -``` -{{< /file >}} - -{{< file path=".gitignore" >}} -```text -# Files and directories that Git should ignore. This is the standard Python -# template covering bytecode, build artifacts, virtual environments, and IDE -# settings. See https://git-scm.com/docs/gitignore for syntax reference. - -# Byte-compiled / optimized / DLL files -__pycache__/ -*.py[cod] -*$py.class - -# C extensions -*.so - -# Distribution / packaging -.Python -build/ -develop-eggs/ -dist/ -downloads/ -eggs/ -.eggs/ -lib/ -lib64/ -parts/ -sdist/ -var/ -wheels/ -share/python-wheels/ -*.egg-info/ -.installed.cfg -*.egg -MANIFEST - -# Unit test / coverage reports -htmlcov/ -.tox/ -.nox/ -.coverage -.coverage.* -.cache -nosetests.xml -coverage.xml -*.cover -*.py,cover -.hypothesis/ -.pytest_cache/ -cover/ - -# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm -__pypackages__/ - -# Environments -.env -.venv -env/ -venv/ -ENV/ -env.bak/ -venv.bak/ - -# Secrets -db/password.txt -``` -{{< /file >}} - -{{< /files >}} - -To learn more about each file, see the following: - -- [Dockerfile](/reference/dockerfile.md) -- [.dockerignore](/reference/dockerfile.md#dockerignore-file) -- [compose.yaml](/reference/compose-file/_index.md) - -## Run the application - -Inside the `python-docker-example` directory, run the following command in a -terminal. - -```console -$ docker compose up --build -``` - -Open a browser and view the application at [http://localhost:8000](http://localhost:8000). You should see a simple FastAPI application. - -In the terminal, press `ctrl`+`c` to stop the application. - -### Run the application in the background - -You can run the application detached from the terminal by adding the `-d` -option. Inside the `python-docker-example` directory, run the following command -in a terminal. - -```console -$ docker compose up --build -d -``` - -Open a browser and view the application at [http://localhost:8000](http://localhost:8000). - -To see the OpenAPI docs you can go to [http://localhost:8000/docs](http://localhost:8000/docs). - -You should see a simple FastAPI application. - -In the terminal, run the following command to stop the application. - -```console -$ docker compose down -``` - -For more information about Compose commands, see the [Compose CLI -reference](/reference/cli/docker/compose/). - -## Summary - -In this section, you learned how you can containerize and run your Python -application using Docker. - -Related information: - -- [Docker Hardened Images](/dhi/) -- [Dockerfile reference](/reference/dockerfile.md) -- [Multi-stage builds](/manuals/build/building/multi-stage.md) -- [Docker Compose overview](/manuals/compose/_index.md) - -## Next steps - -In the next section, you'll take a look at how to set up a local development environment using Docker containers. diff --git a/content/guides/python/deploy.md b/content/guides/python/deploy.md deleted file mode 100644 index a47ef1213127..000000000000 --- a/content/guides/python/deploy.md +++ /dev/null @@ -1,341 +0,0 @@ ---- -title: Test your Python deployment -linkTitle: Test your deployment -weight: 50 -keywords: deploy, kubernetes, python -description: Learn how to develop locally using Kubernetes -aliases: - - /language/python/deploy/ - - /guides/language/python/deploy/ ---- - -## Prerequisites - -- Complete all the previous sections of this guide, starting with [Use containers for Python development](develop.md). -- [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. - -## Overview - -[Kubernetes](https://kubernetes.io/) is an open source platform that runs and -orchestrates container workloads across one or more machines. You describe -what you want to run, like which container images, how many replicas, and -which network ports to expose, in YAML manifest files. Kubernetes reads the -manifests and makes the cluster match that description. - -In this section, you'll use the Kubernetes environment built into Docker -Desktop to deploy your application locally. You'll write two manifest files, -one for the PostgreSQL database and one for the FastAPI application, apply -them with `kubectl`, and verify the deployment by hitting your application -from a terminal. - -## Registry authentication - -The Docker Hardened Images used in this guide are hosted on `dhi.io`. Docker -Desktop's Kubernetes shares credentials with Docker Desktop, so the `docker login dhi.io` -you completed earlier is all that's needed. No additional image pull secret is required. - -> [!NOTE] -> -> If you're deploying to a Kubernetes cluster outside of Docker Desktop, you'll -> need to create an image pull secret and reference it in your pod specs. See -> [Use a Docker Hardened Image](/dhi/how-to/use/#use-with-kubernetes) for instructions. - -## Create a Kubernetes YAML file - -Create the following two Kubernetes manifest files in your -`python-docker-example` directory. Before applying -`docker-python-kubernetes.yaml`, replace `DOCKER_USERNAME/REPO_NAME` with your -Docker username and the repository name that you created in [Configure CI/CD for -your Python application](./configure-github-actions.md). - -{{< files name="python-docker-example" >}} - -{{< file path="docker-postgres-kubernetes.yaml" status="new" >}} -```yaml -# Kubernetes manifests for the PostgreSQL database used by the FastAPI app. -# Contains a Deployment, Service, PersistentVolumeClaim, and Secret. - -# Deployment: runs one PostgreSQL pod. The image, port, env vars, and the -# persistent volume mount are all defined here. -apiVersion: apps/v1 -kind: Deployment -metadata: - name: postgres - namespace: default -spec: - replicas: 1 - selector: - matchLabels: - app: postgres - template: - metadata: - labels: - app: postgres - spec: - containers: - - name: postgres - image: dhi.io/postgres:18 - ports: - - containerPort: 5432 - env: - - name: POSTGRES_DB - value: example - - name: POSTGRES_USER - value: postgres - - name: POSTGRES_PASSWORD - valueFrom: - secretKeyRef: - name: postgres-secret - key: POSTGRES_PASSWORD - volumeMounts: - - name: postgres-data - mountPath: /var/lib/postgresql - volumes: - - name: postgres-data - persistentVolumeClaim: - claimName: postgres-pvc ---- -# Service: exposes PostgreSQL inside the cluster on port 5432 so the -# application pod can reach it by the DNS name `postgres`. -apiVersion: v1 -kind: Service -metadata: - name: postgres - namespace: default -spec: - ports: - - port: 5432 - selector: - app: postgres ---- -# PersistentVolumeClaim: storage that survives pod restarts. -apiVersion: v1 -kind: PersistentVolumeClaim -metadata: - name: postgres-pvc - namespace: default -spec: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 1Gi ---- -# Secret: holds the database password (base64-encoded). Referenced by both -# the postgres Deployment and the application Deployment. -apiVersion: v1 -kind: Secret -metadata: - name: postgres-secret - namespace: default -type: Opaque -data: - POSTGRES_PASSWORD: cG9zdGdyZXNfcGFzc3dvcmQ= # Base64 encoded password (e.g., 'postgres_password') -``` -{{< /file >}} - -{{< file path="docker-python-kubernetes.yaml" status="new" >}} -```yaml -# Kubernetes manifests for the FastAPI application. -# Contains a Deployment and a NodePort Service. - -# Deployment: runs the FastAPI app. Connection details to the postgres -# service are passed in via environment variables, and the database -# password comes from the shared postgres-secret. -apiVersion: apps/v1 -kind: Deployment -metadata: - name: docker-python-demo - namespace: default -spec: - replicas: 1 - selector: - matchLabels: - service: fastapi - template: - metadata: - labels: - service: fastapi - spec: - containers: - - name: fastapi-service - image: DOCKER_USERNAME/REPO_NAME - imagePullPolicy: Always - env: - - name: POSTGRES_PASSWORD - valueFrom: - secretKeyRef: - name: postgres-secret - key: POSTGRES_PASSWORD - - name: POSTGRES_USER - value: postgres - - name: POSTGRES_DB - value: example - - name: POSTGRES_SERVER - value: postgres - - name: POSTGRES_PORT - value: "5432" - ports: - - containerPort: 8000 ---- -# Service: exposes the FastAPI app on port 30001 of the cluster node so -# you can reach it from your host with `curl http://localhost:30001/`. -apiVersion: v1 -kind: Service -metadata: - name: service-entrypoint - namespace: default -spec: - type: NodePort - selector: - service: fastapi - ports: - - port: 8000 - targetPort: 8000 - nodePort: 30001 -``` -{{< /file >}} - -{{< /files >}} - -In these Kubernetes YAML files, there are various objects, separated by the `---`: - -- A Deployment, describing a scalable group of identical pods. In this case, - you'll get just one replica, or copy of your pod. That pod, which is - described under `template`, has just one container in it. The - container is created from the image built by GitHub Actions in [Configure CI/CD for - your Python application](configure-github-actions.md). -- A Service, which will define how the ports are mapped in the containers. -- A PersistentVolumeClaim, to define a storage that will be persistent through restarts for the database. -- A Secret, which stores the database password as a Kubernetes Secret resource. -- A NodePort service, which will route traffic from port 30001 on your host to - port 8000 inside the pods it routes to, so you can reach your app - from the network. - -To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). - -> [!NOTE] -> -> The `NodePort` service is good for development and testing. For production, implement an [ingress controller](https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/) instead. - -## Deploy and check your application - -1. In a terminal, navigate to `python-docker-example` and deploy your database to - Kubernetes. - - ```console - $ kubectl apply -f docker-postgres-kubernetes.yaml - ``` - - You should see output that looks like the following, indicating your Kubernetes objects were created successfully. - - ```console - deployment.apps/postgres created - service/postgres created - persistentvolumeclaim/postgres-pvc created - secret/postgres-secret created - ``` - - Now, deploy your Python application. - - ```console - $ kubectl apply -f docker-python-kubernetes.yaml - ``` - - You should see output that looks like the following, indicating your Kubernetes objects were created successfully. - - ```console - deployment.apps/docker-python-demo created - service/service-entrypoint created - ``` - -2. Make sure everything worked by listing your deployments. - - ```console - $ kubectl get deployments - ``` - - Your deployment should be listed as follows: - - ```console - NAME READY UP-TO-DATE AVAILABLE AGE - docker-python-demo 1/1 1 1 48s - postgres 1/1 1 1 2m39s - ``` - - This indicates all one of the pods you asked for in your YAML are up and running. Do the same check for your services. - - ```console - $ kubectl get services - ``` - - You should get output like the following. - - ```console - NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE - kubernetes ClusterIP 10.43.0.1 443/TCP 13h - postgres ClusterIP 10.43.209.25 5432/TCP 3m10s - service-entrypoint NodePort 10.43.67.120 8000:30001/TCP 79s - ``` - - In addition to the default `kubernetes` service, you can see your `service-entrypoint` service, accepting traffic on port 30001/TCP and the internal `ClusterIP` `postgres` with the port `5432` open to accept connections from your Python app. - -3. In a terminal, curl the root endpoint to verify the application is running. - - ```console - $ curl http://localhost:30001/ - Hello, Docker! - ``` - -4. Exercise the database by creating a hero with a POST request: - - ```console - $ curl -X 'POST' \ - 'http://localhost:30001/heroes/' \ - -H 'accept: application/json' \ - -H 'Content-Type: application/json' \ - -d '{ - "id": 1, - "name": "my hero", - "secret_name": "austing", - "age": 12 - }' - ``` - - You should receive the following response: - - ```json - { - "age": 12, - "id": 1, - "name": "my hero", - "secret_name": "austing" - } - ``` - - Then read it back with a GET request: - - ```console - $ curl http://localhost:30001/heroes/ - ``` - - You should receive an array containing the hero you just created. This - confirms the application can read from and write to the PostgreSQL database - running in the cluster. - -5. Run the following commands to tear down your application. - - ```console - $ kubectl delete -f docker-python-kubernetes.yaml - $ kubectl delete -f docker-postgres-kubernetes.yaml - ``` - -## Summary - -In this section, you learned how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. - -Related information: - -- [Kubernetes documentation](https://kubernetes.io/docs/home/) -- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) -- [Use a Docker Hardened Image with Kubernetes](/dhi/how-to/use/#use-with-kubernetes) diff --git a/content/guides/python/develop.md b/content/guides/python/develop.md deleted file mode 100644 index 7cbfc9f378a5..000000000000 --- a/content/guides/python/develop.md +++ /dev/null @@ -1,1560 +0,0 @@ ---- -title: Use containers for Python development -linkTitle: Develop your app -weight: 15 -keywords: python, local, development -description: Learn how to develop your Python application locally. -aliases: - - /language/python/develop/ - - /guides/language/python/develop/ ---- - -## Prerequisites - -Complete [Containerize a Python application](containerize.md). - -## Overview - -Once your application runs in a container, the next step is making the -container loop part of your everyday development workflow. Code changes should -show up quickly, and services your app depends on, like databases, should run -right alongside it. - -In this section, you'll extend the project from the previous topic by adding a -PostgreSQL database service to your `compose.yaml`, persisting the database -data in a named volume, and enabling Compose Watch so that changes you save in -your editor are picked up by the running container without a manual rebuild. - -## Update the application - -You'll update your application to connect to a PostgreSQL database. Continue -working in your `python-docker-example` directory. - -Replace `app.py` and `requirements.txt`, and add a new `config.py` file with the -following contents. - -> [!NOTE] -> -> The application won't run yet after this step. It tries to connect to a -> PostgreSQL database that doesn't exist. The next two sections add the -> database service and the Docker configuration needed to run everything -> together. - -{{< files name="python-docker-example" >}} - -{{< file path="app.py" status="modified" >}} -```python -# FastAPI application backed by a PostgreSQL database via SQLModel. -# The FastAPI lifespan handler creates database tables at startup. -# Endpoints: GET / (greeting), POST /heroes/ (create), GET /heroes/ (list). -# See https://fastapi.tiangolo.com/ and https://sqlmodel.tiangolo.com/ - -from collections.abc import AsyncGenerator, Sequence -from contextlib import asynccontextmanager - -from fastapi import FastAPI -from sqlmodel import Field, Session, SQLModel, create_engine, select - -from config import settings - - -class Hero(SQLModel, table=True): - id: int | None = Field(default=None, primary_key=True) - name: str = Field(index=True) - secret_name: str - age: int | None = Field(default=None, index=True) - - -engine = create_engine(str(settings.SQLALCHEMY_DATABASE_URI)) - - -def create_db_and_tables() -> None: - SQLModel.metadata.create_all(engine) - - -@asynccontextmanager -async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]: - create_db_and_tables() - yield - - -app = FastAPI(lifespan=lifespan) - - -@app.get("/") -def hello() -> str: - return "Hello, Docker!" - - -@app.post("/heroes/") -def create_hero(hero: Hero) -> Hero: - with Session(engine) as session: - session.add(hero) - session.commit() - session.refresh(hero) - return hero - - -@app.get("/heroes/") -def read_heroes() -> Sequence[Hero]: - with Session(engine) as session: - heroes = session.exec(select(Hero)).all() - return heroes -``` -{{< /file >}} - -{{< file path="config.py" status="new" >}} -```python -# Pydantic settings that read PostgreSQL connection details from the -# environment. Supports a password file (Docker secrets) via -# POSTGRES_PASSWORD_FILE in addition to POSTGRES_PASSWORD. -# See https://docs.pydantic.dev/latest/concepts/pydantic_settings/ - -import os -from typing import Any - -from pydantic import ( - PostgresDsn, - computed_field, - field_validator, - model_validator, -) -from pydantic_core import MultiHostUrl -from pydantic_settings import BaseSettings - - -class Settings(BaseSettings): - POSTGRES_SERVER: str - POSTGRES_PORT: int = 5432 - POSTGRES_USER: str - POSTGRES_PASSWORD: str | None = None - POSTGRES_PASSWORD_FILE: str | None = None - POSTGRES_DB: str - - @model_validator(mode="before") - @classmethod - def check_postgres_password(cls, data: Any) -> Any: - """Validate that either POSTGRES_PASSWORD or POSTGRES_PASSWORD_FILE is set.""" - if isinstance(data, dict): - password_file: str | None = data.get("POSTGRES_PASSWORD_FILE") # type: ignore - password: str | None = data.get("POSTGRES_PASSWORD") # type: ignore - if password_file is None and password is None: - raise ValueError( - "At least one of POSTGRES_PASSWORD_FILE and POSTGRES_PASSWORD must be set." - ) - return data # type: ignore - - @field_validator("POSTGRES_PASSWORD_FILE", mode="before") - @classmethod - def read_password_from_file(cls, v: str | None) -> str | None: - if v is not None: - file_path = v - if os.path.exists(file_path): - with open(file_path) as file: - return file.read().strip() - raise ValueError(f"Password file {file_path} does not exist.") - return v - - @computed_field - @property - def SQLALCHEMY_DATABASE_URI(self) -> PostgresDsn: - url = MultiHostUrl.build( - scheme="postgresql+psycopg", - username=self.POSTGRES_USER, - password=self.POSTGRES_PASSWORD - if self.POSTGRES_PASSWORD - else self.POSTGRES_PASSWORD_FILE, - host=self.POSTGRES_SERVER, - port=self.POSTGRES_PORT, - path=self.POSTGRES_DB, - ) - return PostgresDsn(url) - - -settings = Settings() # type: ignore -``` -{{< /file >}} - -{{< file path="requirements.txt" status="modified" hl_lines="5-7" >}} -```text -# Python package dependencies for the application, pinned for reproducible builds. -# See https://pip.pypa.io/en/stable/reference/requirements-file-format/ - -fastapi==0.115.12 -sqlmodel==0.0.24 -psycopg[binary]==3.2.9 -pydantic-settings==2.9.1 -uvicorn==0.34.3 -``` -{{< /file >}} - -{{< file path="Dockerfile" >}} -```dockerfile -# syntax=docker/dockerfile:1 - -# Comments are provided throughout this file to help you get started. -# If you need more help, visit the Dockerfile reference guide at -# https://docs.docker.com/go/dockerfile-reference/ - -# This Dockerfile uses Docker Hardened Images (DHI) for enhanced security. -# For more information, see https://docs.docker.com/dhi/ - -# Use the dev image to build and install dependencies. -FROM dhi.io/python:3.12-dev AS builder - -WORKDIR /app - -RUN python3 -m venv /venv -ENV PATH="/venv/bin:$PATH" - -# Download dependencies as a separate step to take advantage of Docker's caching. -# Leverage a cache mount to /root/.cache/pip to speed up subsequent builds. -# Leverage a bind mount to requirements.txt to avoid having to copy them into -# this layer. -RUN --mount=type=cache,target=/root/.cache/pip \ - --mount=type=bind,source=requirements.txt,target=requirements.txt \ - pip install -r requirements.txt - -# Use the minimal runtime image. It runs as nonroot by default. -FROM dhi.io/python:3.12 - -WORKDIR /app - -COPY --from=builder /venv /venv -ENV PATH="/venv/bin:$PATH" - -# Copy the source code into the container. -COPY . . - -# Expose the port that the application listens on. -EXPOSE 8000 - -# Run the application. -CMD ["/venv/bin/python3", "-m", "uvicorn", "app:app", "--host=0.0.0.0", "--port=8000"] -``` -{{< /file >}} - -{{< file path="compose.yaml" >}} -```yaml -# Comments are provided throughout this file to help you get started. -# If you need more help, visit the Docker Compose reference guide at -# https://docs.docker.com/go/compose-spec-reference/ - -# Here the instructions define your application as a service called "server". -# This service is built from the Dockerfile in the current directory. -# You can add other services your application may depend on here, such as a -# database or a cache. For examples, see the Awesome Compose repository: -# https://github.com/docker/awesome-compose -services: - server: - build: - context: . - ports: - - 8000:8000 -``` -{{< /file >}} - -{{< file path=".dockerignore" >}} -```text -# Include any files or directories that you don't want to be copied to your -# container here (e.g., local build artifacts, temporary files, etc.). -# -# For more help, visit the .dockerignore file reference guide at -# https://docs.docker.com/go/build-context-dockerignore/ - -**/.DS_Store -**/__pycache__ -**/.venv -**/.classpath -**/.dockerignore -**/.env -**/.git -**/.gitignore -**/.project -**/.settings -**/.toolstarget -**/.vs -**/.vscode -**/*.*proj.user -**/*.dbmdl -**/*.jfm -**/bin -**/charts -**/docker-compose* -**/compose.y*ml -**/Dockerfile* -**/node_modules -**/npm-debug.log -**/obj -**/secrets.dev.yaml -**/values.dev.yaml -LICENSE -README.md -``` -{{< /file >}} - -{{< file path=".gitignore" >}} -```text -# Files and directories that Git should ignore. This is the standard Python -# template covering bytecode, build artifacts, virtual environments, and IDE -# settings. See https://git-scm.com/docs/gitignore for syntax reference. - -# Byte-compiled / optimized / DLL files -__pycache__/ -*.py[cod] -*$py.class - -# C extensions -*.so - -# Distribution / packaging -.Python -build/ -develop-eggs/ -dist/ -downloads/ -eggs/ -.eggs/ -lib/ -lib64/ -parts/ -sdist/ -var/ -wheels/ -share/python-wheels/ -*.egg-info/ -.installed.cfg -*.egg -MANIFEST - -# Unit test / coverage reports -htmlcov/ -.tox/ -.nox/ -.coverage -.coverage.* -.cache -nosetests.xml -coverage.xml -*.cover -*.py,cover -.hypothesis/ -.pytest_cache/ -cover/ - -# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm -__pypackages__/ - -# Environments -.env -.venv -env/ -venv/ -ENV/ -env.bak/ -venv.bak/ - -# Secrets -db/password.txt -``` -{{< /file >}} - -{{< /files >}} - -## Update Docker assets - -Replace `Dockerfile` and `compose.yaml` with the following. - -{{< files name="python-docker-example" >}} - -{{< file path="app.py" >}} -```python -# FastAPI application backed by a PostgreSQL database via SQLModel. -# The FastAPI lifespan handler creates database tables at startup. -# Endpoints: GET / (greeting), POST /heroes/ (create), GET /heroes/ (list). -# See https://fastapi.tiangolo.com/ and https://sqlmodel.tiangolo.com/ - -from collections.abc import AsyncGenerator, Sequence -from contextlib import asynccontextmanager - -from fastapi import FastAPI -from sqlmodel import Field, Session, SQLModel, create_engine, select - -from config import settings - - -class Hero(SQLModel, table=True): - id: int | None = Field(default=None, primary_key=True) - name: str = Field(index=True) - secret_name: str - age: int | None = Field(default=None, index=True) - - -engine = create_engine(str(settings.SQLALCHEMY_DATABASE_URI)) - - -def create_db_and_tables() -> None: - SQLModel.metadata.create_all(engine) - - -@asynccontextmanager -async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]: - create_db_and_tables() - yield - - -app = FastAPI(lifespan=lifespan) - - -@app.get("/") -def hello() -> str: - return "Hello, Docker!" - - -@app.post("/heroes/") -def create_hero(hero: Hero) -> Hero: - with Session(engine) as session: - session.add(hero) - session.commit() - session.refresh(hero) - return hero - - -@app.get("/heroes/") -def read_heroes() -> Sequence[Hero]: - with Session(engine) as session: - heroes = session.exec(select(Hero)).all() - return heroes -``` -{{< /file >}} - -{{< file path="config.py" >}} -```python -# Pydantic settings that read PostgreSQL connection details from the -# environment. Supports a password file (Docker secrets) via -# POSTGRES_PASSWORD_FILE in addition to POSTGRES_PASSWORD. -# See https://docs.pydantic.dev/latest/concepts/pydantic_settings/ - -import os -from typing import Any - -from pydantic import ( - PostgresDsn, - computed_field, - field_validator, - model_validator, -) -from pydantic_core import MultiHostUrl -from pydantic_settings import BaseSettings - - -class Settings(BaseSettings): - POSTGRES_SERVER: str - POSTGRES_PORT: int = 5432 - POSTGRES_USER: str - POSTGRES_PASSWORD: str | None = None - POSTGRES_PASSWORD_FILE: str | None = None - POSTGRES_DB: str - - @model_validator(mode="before") - @classmethod - def check_postgres_password(cls, data: Any) -> Any: - """Validate that either POSTGRES_PASSWORD or POSTGRES_PASSWORD_FILE is set.""" - if isinstance(data, dict): - password_file: str | None = data.get("POSTGRES_PASSWORD_FILE") # type: ignore - password: str | None = data.get("POSTGRES_PASSWORD") # type: ignore - if password_file is None and password is None: - raise ValueError( - "At least one of POSTGRES_PASSWORD_FILE and POSTGRES_PASSWORD must be set." - ) - return data # type: ignore - - @field_validator("POSTGRES_PASSWORD_FILE", mode="before") - @classmethod - def read_password_from_file(cls, v: str | None) -> str | None: - if v is not None: - file_path = v - if os.path.exists(file_path): - with open(file_path) as file: - return file.read().strip() - raise ValueError(f"Password file {file_path} does not exist.") - return v - - @computed_field - @property - def SQLALCHEMY_DATABASE_URI(self) -> PostgresDsn: - url = MultiHostUrl.build( - scheme="postgresql+psycopg", - username=self.POSTGRES_USER, - password=self.POSTGRES_PASSWORD - if self.POSTGRES_PASSWORD - else self.POSTGRES_PASSWORD_FILE, - host=self.POSTGRES_SERVER, - port=self.POSTGRES_PORT, - path=self.POSTGRES_DB, - ) - return PostgresDsn(url) - - -settings = Settings() # type: ignore -``` -{{< /file >}} - -{{< file path="requirements.txt" >}} -```text -# Python package dependencies for the application, pinned for reproducible builds. -# See https://pip.pypa.io/en/stable/reference/requirements-file-format/ - -fastapi==0.115.12 -sqlmodel==0.0.24 -psycopg[binary]==3.2.9 -pydantic-settings==2.9.1 -uvicorn==0.34.3 -``` -{{< /file >}} - -{{< file path="Dockerfile" status="modified" hl_lines="11,27-34,37,45" >}} -```dockerfile -# syntax=docker/dockerfile:1 - -# Comments are provided throughout this file to help you get started. -# If you need more help, visit the Dockerfile reference guide at -# https://docs.docker.com/go/dockerfile-reference/ - -# This Dockerfile uses Docker Hardened Images (DHI) for enhanced security. -# For more information, see https://docs.docker.com/dhi/ - -# Use the dev image to build and install dependencies. -# The builder stage is also used directly in development (see compose.yaml). -FROM dhi.io/python:3.12-dev AS builder - -WORKDIR /app - -RUN python3 -m venv /venv -ENV PATH="/venv/bin:$PATH" - -# Download dependencies as a separate step to take advantage of Docker's caching. -# Leverage a cache mount to /root/.cache/pip to speed up subsequent builds. -# Leverage a bind mount to requirements.txt to avoid having to copy them -# into this layer. -RUN --mount=type=cache,target=/root/.cache/pip \ - --mount=type=bind,source=requirements.txt,target=requirements.txt \ - pip install -r requirements.txt - -# Copy the source code into the container. -COPY . . - -# Expose the port that the application listens on. -EXPOSE 8000 - -# Run the application. -CMD ["/venv/bin/python3", "-m", "uvicorn", "app:app", "--host=0.0.0.0", "--port=8000"] - - -# Use the minimal runtime image for production. It runs as nonroot by default. -FROM dhi.io/python:3.12 - -WORKDIR /app - -COPY --from=builder /venv /venv -ENV PATH="/venv/bin:$PATH" - -COPY --from=builder /app . - -EXPOSE 8000 - -CMD ["/venv/bin/python3", "-m", "uvicorn", "app:app", "--host=0.0.0.0", "--port=8000"] -``` -{{< /file >}} - -{{< file path="compose.yaml" status="modified" hl_lines="8" >}} -```yaml -services: - # Application service. The `target: builder` line builds the development - # image (includes a shell and tools); the production stage of the - # Dockerfile is unused in development. - server: - build: - context: . - target: builder - ports: - - 8000:8000 -``` -{{< /file >}} - -{{< file path=".dockerignore" >}} -```text -# Include any files or directories that you don't want to be copied to your -# container here (e.g., local build artifacts, temporary files, etc.). -# -# For more help, visit the .dockerignore file reference guide at -# https://docs.docker.com/go/build-context-dockerignore/ - -**/.DS_Store -**/__pycache__ -**/.venv -**/.classpath -**/.dockerignore -**/.env -**/.git -**/.gitignore -**/.project -**/.settings -**/.toolstarget -**/.vs -**/.vscode -**/*.*proj.user -**/*.dbmdl -**/*.jfm -**/bin -**/charts -**/docker-compose* -**/compose.y*ml -**/Dockerfile* -**/node_modules -**/npm-debug.log -**/obj -**/secrets.dev.yaml -**/values.dev.yaml -LICENSE -README.md -``` -{{< /file >}} - -{{< file path=".gitignore" >}} -```text -# Files and directories that Git should ignore. This is the standard Python -# template covering bytecode, build artifacts, virtual environments, and IDE -# settings. See https://git-scm.com/docs/gitignore for syntax reference. - -# Byte-compiled / optimized / DLL files -__pycache__/ -*.py[cod] -*$py.class - -# C extensions -*.so - -# Distribution / packaging -.Python -build/ -develop-eggs/ -dist/ -downloads/ -eggs/ -.eggs/ -lib/ -lib64/ -parts/ -sdist/ -var/ -wheels/ -share/python-wheels/ -*.egg-info/ -.installed.cfg -*.egg -MANIFEST - -# Unit test / coverage reports -htmlcov/ -.tox/ -.nox/ -.coverage -.coverage.* -.cache -nosetests.xml -coverage.xml -*.cover -*.py,cover -.hypothesis/ -.pytest_cache/ -cover/ - -# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm -__pypackages__/ - -# Environments -.env -.venv -env/ -venv/ -ENV/ -env.bak/ -venv.bak/ - -# Secrets -db/password.txt -``` -{{< /file >}} - -{{< /files >}} - -### About these changes - -The `Dockerfile` builder stage now includes `COPY . .` and a `CMD` -instruction, which makes it directly runnable. This lets Compose target the -builder stage during development without rebuilding the production stage. The -production stage at the bottom is unchanged and still produces a minimal, -nonroot runtime image for shipping. - -In `compose.yaml`, the new `target: builder` line tells Compose to build and -run the builder stage of the Dockerfile during development. Unlike the minimal -production image, the development image includes a shell and additional tools -that make debugging easier. If you need a shell in a running production -container, use [Docker Debug](/reference/cli/docker/debug/) instead. - -## Add a local database and persist data - -You can use containers to set up local services, like a database. In this -section, you'll update the `compose.yaml` file to define a database service -and a volume to persist data, and add a `db/password.txt` file that holds the -database password. - -{{< files name="python-docker-example" >}} - -{{< file path="app.py" >}} -```python -# FastAPI application backed by a PostgreSQL database via SQLModel. -# The FastAPI lifespan handler creates database tables at startup. -# Endpoints: GET / (greeting), POST /heroes/ (create), GET /heroes/ (list). -# See https://fastapi.tiangolo.com/ and https://sqlmodel.tiangolo.com/ - -from collections.abc import AsyncGenerator, Sequence -from contextlib import asynccontextmanager - -from fastapi import FastAPI -from sqlmodel import Field, Session, SQLModel, create_engine, select - -from config import settings - - -class Hero(SQLModel, table=True): - id: int | None = Field(default=None, primary_key=True) - name: str = Field(index=True) - secret_name: str - age: int | None = Field(default=None, index=True) - - -engine = create_engine(str(settings.SQLALCHEMY_DATABASE_URI)) - - -def create_db_and_tables() -> None: - SQLModel.metadata.create_all(engine) - - -@asynccontextmanager -async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]: - create_db_and_tables() - yield - - -app = FastAPI(lifespan=lifespan) - - -@app.get("/") -def hello() -> str: - return "Hello, Docker!" - - -@app.post("/heroes/") -def create_hero(hero: Hero) -> Hero: - with Session(engine) as session: - session.add(hero) - session.commit() - session.refresh(hero) - return hero - - -@app.get("/heroes/") -def read_heroes() -> Sequence[Hero]: - with Session(engine) as session: - heroes = session.exec(select(Hero)).all() - return heroes -``` -{{< /file >}} - -{{< file path="config.py" >}} -```python -# Pydantic settings that read PostgreSQL connection details from the -# environment. Supports a password file (Docker secrets) via -# POSTGRES_PASSWORD_FILE in addition to POSTGRES_PASSWORD. -# See https://docs.pydantic.dev/latest/concepts/pydantic_settings/ - -import os -from typing import Any - -from pydantic import ( - PostgresDsn, - computed_field, - field_validator, - model_validator, -) -from pydantic_core import MultiHostUrl -from pydantic_settings import BaseSettings - - -class Settings(BaseSettings): - POSTGRES_SERVER: str - POSTGRES_PORT: int = 5432 - POSTGRES_USER: str - POSTGRES_PASSWORD: str | None = None - POSTGRES_PASSWORD_FILE: str | None = None - POSTGRES_DB: str - - @model_validator(mode="before") - @classmethod - def check_postgres_password(cls, data: Any) -> Any: - """Validate that either POSTGRES_PASSWORD or POSTGRES_PASSWORD_FILE is set.""" - if isinstance(data, dict): - password_file: str | None = data.get("POSTGRES_PASSWORD_FILE") # type: ignore - password: str | None = data.get("POSTGRES_PASSWORD") # type: ignore - if password_file is None and password is None: - raise ValueError( - "At least one of POSTGRES_PASSWORD_FILE and POSTGRES_PASSWORD must be set." - ) - return data # type: ignore - - @field_validator("POSTGRES_PASSWORD_FILE", mode="before") - @classmethod - def read_password_from_file(cls, v: str | None) -> str | None: - if v is not None: - file_path = v - if os.path.exists(file_path): - with open(file_path) as file: - return file.read().strip() - raise ValueError(f"Password file {file_path} does not exist.") - return v - - @computed_field - @property - def SQLALCHEMY_DATABASE_URI(self) -> PostgresDsn: - url = MultiHostUrl.build( - scheme="postgresql+psycopg", - username=self.POSTGRES_USER, - password=self.POSTGRES_PASSWORD - if self.POSTGRES_PASSWORD - else self.POSTGRES_PASSWORD_FILE, - host=self.POSTGRES_SERVER, - port=self.POSTGRES_PORT, - path=self.POSTGRES_DB, - ) - return PostgresDsn(url) - - -settings = Settings() # type: ignore -``` -{{< /file >}} - -{{< file path="requirements.txt" >}} -```text -# Python package dependencies for the application, pinned for reproducible builds. -# See https://pip.pypa.io/en/stable/reference/requirements-file-format/ - -fastapi==0.115.12 -sqlmodel==0.0.24 -psycopg[binary]==3.2.9 -pydantic-settings==2.9.1 -uvicorn==0.34.3 -``` -{{< /file >}} - -{{< file path="Dockerfile" >}} -```dockerfile -# syntax=docker/dockerfile:1 - -# Comments are provided throughout this file to help you get started. -# If you need more help, visit the Dockerfile reference guide at -# https://docs.docker.com/go/dockerfile-reference/ - -# This Dockerfile uses Docker Hardened Images (DHI) for enhanced security. -# For more information, see https://docs.docker.com/dhi/ - -# Use the dev image to build and install dependencies. -# The builder stage is also used directly in development (see compose.yaml). -FROM dhi.io/python:3.12-dev AS builder - -WORKDIR /app - -RUN python3 -m venv /venv -ENV PATH="/venv/bin:$PATH" - -# Download dependencies as a separate step to take advantage of Docker's caching. -# Leverage a cache mount to /root/.cache/pip to speed up subsequent builds. -# Leverage a bind mount to requirements.txt to avoid having to copy them -# into this layer. -RUN --mount=type=cache,target=/root/.cache/pip \ - --mount=type=bind,source=requirements.txt,target=requirements.txt \ - pip install -r requirements.txt - -# Copy the source code into the container. -COPY . . - -# Expose the port that the application listens on. -EXPOSE 8000 - -# Run the application. -CMD ["/venv/bin/python3", "-m", "uvicorn", "app:app", "--host=0.0.0.0", "--port=8000"] - - -# Use the minimal runtime image for production. It runs as nonroot by default. -FROM dhi.io/python:3.12 - -WORKDIR /app - -COPY --from=builder /venv /venv -ENV PATH="/venv/bin:$PATH" - -COPY --from=builder /app . - -EXPOSE 8000 - -CMD ["/venv/bin/python3", "-m", "uvicorn", "app:app", "--host=0.0.0.0", "--port=8000"] -``` -{{< /file >}} - -{{< file path="compose.yaml" status="modified" hl_lines="11-46" >}} -```yaml -services: - # Application service. The `target: builder` line builds the development - # image (includes a shell and tools); the production stage of the - # Dockerfile is unused in development. - server: - build: - context: . - target: builder - ports: - - 8000:8000 - environment: - - POSTGRES_SERVER=db - - POSTGRES_USER=postgres - - POSTGRES_DB=example - - POSTGRES_PASSWORD_FILE=/run/secrets/db-password - depends_on: - db: - condition: service_healthy - secrets: - - db-password - # Database service. Reads the password from a Docker secret mounted at - # /run/secrets/db-password. Compose waits for the healthcheck to pass - # before starting the server, via the server's depends_on. - db: - image: dhi.io/postgres:18 - restart: always - user: postgres - secrets: - - db-password - volumes: - - db-data:/var/lib/postgresql - environment: - - POSTGRES_DB=example - - POSTGRES_PASSWORD_FILE=/run/secrets/db-password - expose: - - 5432 - healthcheck: - test: ["CMD", "pg_isready"] - interval: 10s - timeout: 5s - retries: 5 -volumes: - db-data: -secrets: - db-password: - file: db/password.txt -``` -{{< /file >}} - -{{< file path="db/password.txt" status="new" >}} -```text -mysecretpassword -``` -{{< /file >}} - -{{< file path=".dockerignore" >}} -```text -# Include any files or directories that you don't want to be copied to your -# container here (e.g., local build artifacts, temporary files, etc.). -# -# For more help, visit the .dockerignore file reference guide at -# https://docs.docker.com/go/build-context-dockerignore/ - -**/.DS_Store -**/__pycache__ -**/.venv -**/.classpath -**/.dockerignore -**/.env -**/.git -**/.gitignore -**/.project -**/.settings -**/.toolstarget -**/.vs -**/.vscode -**/*.*proj.user -**/*.dbmdl -**/*.jfm -**/bin -**/charts -**/docker-compose* -**/compose.y*ml -**/Dockerfile* -**/node_modules -**/npm-debug.log -**/obj -**/secrets.dev.yaml -**/values.dev.yaml -LICENSE -README.md -``` -{{< /file >}} - -{{< file path=".gitignore" >}} -```text -# Files and directories that Git should ignore. This is the standard Python -# template covering bytecode, build artifacts, virtual environments, and IDE -# settings. See https://git-scm.com/docs/gitignore for syntax reference. - -# Byte-compiled / optimized / DLL files -__pycache__/ -*.py[cod] -*$py.class - -# C extensions -*.so - -# Distribution / packaging -.Python -build/ -develop-eggs/ -dist/ -downloads/ -eggs/ -.eggs/ -lib/ -lib64/ -parts/ -sdist/ -var/ -wheels/ -share/python-wheels/ -*.egg-info/ -.installed.cfg -*.egg -MANIFEST - -# Unit test / coverage reports -htmlcov/ -.tox/ -.nox/ -.coverage -.coverage.* -.cache -nosetests.xml -coverage.xml -*.cover -*.py,cover -.hypothesis/ -.pytest_cache/ -cover/ - -# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm -__pypackages__/ - -# Environments -.env -.venv -env/ -venv/ -ENV/ -env.bak/ -venv.bak/ - -# Secrets -db/password.txt -``` -{{< /file >}} - -{{< /files >}} - -> [!NOTE] -> -> To learn more about the instructions in the Compose file, see [Compose file -> reference](/reference/compose-file/). - -Now, run the following `docker compose up` command to start your application. - -```console -$ docker compose up --build -``` - -Now test your API endpoint. Open a new terminal then make a request to the server using the curl commands: - -Create an object with a POST request: - -```console -$ curl -X 'POST' \ - 'http://localhost:8000/heroes/' \ - -H 'accept: application/json' \ - -H 'Content-Type: application/json' \ - -d '{ - "id": 1, - "name": "my hero", - "secret_name": "austing", - "age": 12 -}' -``` - -You should receive the following response: - -```json -{ - "age": 12, - "id": 1, - "name": "my hero", - "secret_name": "austing" -} -``` - -Now make a GET request: - -```console -$ curl -X 'GET' \ - 'http://localhost:8000/heroes/' \ - -H 'accept: application/json' -``` - -You should receive the same response as above because it's the only object in the database. - -```json -{ - "age": 12, - "id": 1, - "name": "my hero", - "secret_name": "austing" -} -``` - -Press `ctrl+c` in the terminal to stop your application. - -## Automatically update services - -Use Compose Watch to automatically update your running Compose services as you -edit and save your code. For more details about Compose Watch, see [Use Compose -Watch](/manuals/compose/how-tos/file-watch.md). - -Open your `compose.yaml` file in an IDE or text editor and add the highlighted -Compose Watch instructions. - -{{< files name="python-docker-example" >}} - -{{< file path="app.py" >}} -```python -# FastAPI application backed by a PostgreSQL database via SQLModel. -# The FastAPI lifespan handler creates database tables at startup. -# Endpoints: GET / (greeting), POST /heroes/ (create), GET /heroes/ (list). -# See https://fastapi.tiangolo.com/ and https://sqlmodel.tiangolo.com/ - -from collections.abc import AsyncGenerator, Sequence -from contextlib import asynccontextmanager - -from fastapi import FastAPI -from sqlmodel import Field, Session, SQLModel, create_engine, select - -from config import settings - - -class Hero(SQLModel, table=True): - id: int | None = Field(default=None, primary_key=True) - name: str = Field(index=True) - secret_name: str - age: int | None = Field(default=None, index=True) - - -engine = create_engine(str(settings.SQLALCHEMY_DATABASE_URI)) - - -def create_db_and_tables() -> None: - SQLModel.metadata.create_all(engine) - - -@asynccontextmanager -async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]: - create_db_and_tables() - yield - - -app = FastAPI(lifespan=lifespan) - - -@app.get("/") -def hello() -> str: - return "Hello, Docker!" - - -@app.post("/heroes/") -def create_hero(hero: Hero) -> Hero: - with Session(engine) as session: - session.add(hero) - session.commit() - session.refresh(hero) - return hero - - -@app.get("/heroes/") -def read_heroes() -> Sequence[Hero]: - with Session(engine) as session: - heroes = session.exec(select(Hero)).all() - return heroes -``` -{{< /file >}} - -{{< file path="config.py" >}} -```python -# Pydantic settings that read PostgreSQL connection details from the -# environment. Supports a password file (Docker secrets) via -# POSTGRES_PASSWORD_FILE in addition to POSTGRES_PASSWORD. -# See https://docs.pydantic.dev/latest/concepts/pydantic_settings/ - -import os -from typing import Any - -from pydantic import ( - PostgresDsn, - computed_field, - field_validator, - model_validator, -) -from pydantic_core import MultiHostUrl -from pydantic_settings import BaseSettings - - -class Settings(BaseSettings): - POSTGRES_SERVER: str - POSTGRES_PORT: int = 5432 - POSTGRES_USER: str - POSTGRES_PASSWORD: str | None = None - POSTGRES_PASSWORD_FILE: str | None = None - POSTGRES_DB: str - - @model_validator(mode="before") - @classmethod - def check_postgres_password(cls, data: Any) -> Any: - """Validate that either POSTGRES_PASSWORD or POSTGRES_PASSWORD_FILE is set.""" - if isinstance(data, dict): - password_file: str | None = data.get("POSTGRES_PASSWORD_FILE") # type: ignore - password: str | None = data.get("POSTGRES_PASSWORD") # type: ignore - if password_file is None and password is None: - raise ValueError( - "At least one of POSTGRES_PASSWORD_FILE and POSTGRES_PASSWORD must be set." - ) - return data # type: ignore - - @field_validator("POSTGRES_PASSWORD_FILE", mode="before") - @classmethod - def read_password_from_file(cls, v: str | None) -> str | None: - if v is not None: - file_path = v - if os.path.exists(file_path): - with open(file_path) as file: - return file.read().strip() - raise ValueError(f"Password file {file_path} does not exist.") - return v - - @computed_field - @property - def SQLALCHEMY_DATABASE_URI(self) -> PostgresDsn: - url = MultiHostUrl.build( - scheme="postgresql+psycopg", - username=self.POSTGRES_USER, - password=self.POSTGRES_PASSWORD - if self.POSTGRES_PASSWORD - else self.POSTGRES_PASSWORD_FILE, - host=self.POSTGRES_SERVER, - port=self.POSTGRES_PORT, - path=self.POSTGRES_DB, - ) - return PostgresDsn(url) - - -settings = Settings() # type: ignore -``` -{{< /file >}} - -{{< file path="requirements.txt" >}} -```text -# Python package dependencies for the application, pinned for reproducible builds. -# See https://pip.pypa.io/en/stable/reference/requirements-file-format/ - -fastapi==0.115.12 -sqlmodel==0.0.24 -psycopg[binary]==3.2.9 -pydantic-settings==2.9.1 -uvicorn==0.34.3 -``` -{{< /file >}} - -{{< file path="Dockerfile" >}} -```dockerfile -# syntax=docker/dockerfile:1 - -# Comments are provided throughout this file to help you get started. -# If you need more help, visit the Dockerfile reference guide at -# https://docs.docker.com/go/dockerfile-reference/ - -# This Dockerfile uses Docker Hardened Images (DHI) for enhanced security. -# For more information, see https://docs.docker.com/dhi/ - -# Use the dev image to build and install dependencies. -# The builder stage is also used directly in development (see compose.yaml). -FROM dhi.io/python:3.12-dev AS builder - -WORKDIR /app - -RUN python3 -m venv /venv -ENV PATH="/venv/bin:$PATH" - -# Download dependencies as a separate step to take advantage of Docker's caching. -# Leverage a cache mount to /root/.cache/pip to speed up subsequent builds. -# Leverage a bind mount to requirements.txt to avoid having to copy them -# into this layer. -RUN --mount=type=cache,target=/root/.cache/pip \ - --mount=type=bind,source=requirements.txt,target=requirements.txt \ - pip install -r requirements.txt - -# Copy the source code into the container. -COPY . . - -# Expose the port that the application listens on. -EXPOSE 8000 - -# Run the application. -CMD ["/venv/bin/python3", "-m", "uvicorn", "app:app", "--host=0.0.0.0", "--port=8000"] - - -# Use the minimal runtime image for production. It runs as nonroot by default. -FROM dhi.io/python:3.12 - -WORKDIR /app - -COPY --from=builder /venv /venv -ENV PATH="/venv/bin:$PATH" - -COPY --from=builder /app . - -EXPOSE 8000 - -CMD ["/venv/bin/python3", "-m", "uvicorn", "app:app", "--host=0.0.0.0", "--port=8000"] -``` -{{< /file >}} - -{{< file path="compose.yaml" status="modified" hl_lines="21-24" >}} -```yaml -services: - # Application service. The `target: builder` line builds the development - # image (includes a shell and tools); the production stage of the - # Dockerfile is unused in development. - server: - build: - context: . - target: builder - ports: - - 8000:8000 - environment: - - POSTGRES_SERVER=db - - POSTGRES_USER=postgres - - POSTGRES_DB=example - - POSTGRES_PASSWORD_FILE=/run/secrets/db-password - depends_on: - db: - condition: service_healthy - secrets: - - db-password - develop: - watch: - - action: rebuild - path: . - db: - image: dhi.io/postgres:18 - restart: always - user: postgres - secrets: - - db-password - volumes: - - db-data:/var/lib/postgresql - environment: - - POSTGRES_DB=example - - POSTGRES_PASSWORD_FILE=/run/secrets/db-password - expose: - - 5432 - healthcheck: - test: ["CMD", "pg_isready"] - interval: 10s - timeout: 5s - retries: 5 -volumes: - db-data: -secrets: - db-password: - file: db/password.txt -``` -{{< /file >}} - -{{< file path="db/password.txt" >}} -```text -mysecretpassword -``` -{{< /file >}} - -{{< file path=".dockerignore" >}} -```text -# Include any files or directories that you don't want to be copied to your -# container here (e.g., local build artifacts, temporary files, etc.). -# -# For more help, visit the .dockerignore file reference guide at -# https://docs.docker.com/go/build-context-dockerignore/ - -**/.DS_Store -**/__pycache__ -**/.venv -**/.classpath -**/.dockerignore -**/.env -**/.git -**/.gitignore -**/.project -**/.settings -**/.toolstarget -**/.vs -**/.vscode -**/*.*proj.user -**/*.dbmdl -**/*.jfm -**/bin -**/charts -**/docker-compose* -**/compose.y*ml -**/Dockerfile* -**/node_modules -**/npm-debug.log -**/obj -**/secrets.dev.yaml -**/values.dev.yaml -LICENSE -README.md -``` -{{< /file >}} - -{{< file path=".gitignore" >}} -```text -# Files and directories that Git should ignore. This is the standard Python -# template covering bytecode, build artifacts, virtual environments, and IDE -# settings. See https://git-scm.com/docs/gitignore for syntax reference. - -# Byte-compiled / optimized / DLL files -__pycache__/ -*.py[cod] -*$py.class - -# C extensions -*.so - -# Distribution / packaging -.Python -build/ -develop-eggs/ -dist/ -downloads/ -eggs/ -.eggs/ -lib/ -lib64/ -parts/ -sdist/ -var/ -wheels/ -share/python-wheels/ -*.egg-info/ -.installed.cfg -*.egg -MANIFEST - -# Unit test / coverage reports -htmlcov/ -.tox/ -.nox/ -.coverage -.coverage.* -.cache -nosetests.xml -coverage.xml -*.cover -*.py,cover -.hypothesis/ -.pytest_cache/ -cover/ - -# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm -__pypackages__/ - -# Environments -.env -.venv -env/ -venv/ -ENV/ -env.bak/ -venv.bak/ - -# Secrets -db/password.txt -``` -{{< /file >}} - -{{< /files >}} - -Run the following command to run your application with Compose Watch. - -```console -$ docker compose watch -``` - -In a terminal, curl the application to get a response. - -```console -$ curl http://localhost:8000 -Hello, Docker! -``` - -Any changes to the application's source files on your local machine will now be immediately reflected in the running container. - -Open `python-docker-example/app.py` in an IDE or text editor and update the `Hello, Docker!` string by adding a few more exclamation marks. - -```diff -- return 'Hello, Docker!' -+ return 'Hello, Docker!!!' -``` - -Save the changes to `app.py` and then wait a few seconds for the application to rebuild. Curl the application again and verify that the updated text appears. - -```console -$ curl http://localhost:8000 -Hello, Docker!!! -``` - -Press `ctrl+c` in the terminal to stop your application. - -## Summary - -In this section, you took a look at setting up your Compose file to add a local -database and persist data. You also learned how to use Compose Watch to automatically rebuild and run your container when you update your code. - -Related information: - -- [Compose file reference](/reference/compose-file/) -- [Compose secrets](/reference/compose-file/secrets.md) -- [Compose Watch](/manuals/compose/how-tos/file-watch.md) -- [Multi-stage builds](/manuals/build/building/multi-stage.md) - -## Next steps - -In the next section, you'll learn how you can set up linting, formatting, and type checking to follow the best practices in Python apps. diff --git a/content/guides/python/lint-format-typing.md b/content/guides/python/lint-format-typing.md deleted file mode 100644 index 64e5b3c5b080..000000000000 --- a/content/guides/python/lint-format-typing.md +++ /dev/null @@ -1,196 +0,0 @@ ---- -title: Linting, formatting, and type checking for Python -linkTitle: Linting and typing -weight: 25 -keywords: Python, linting, formatting, type checking, ruff, pyright -description: Learn how to set up linting, formatting and type checking for your Python application. -aliases: - - /language/python/lint-format-typing/ ---- - -## Prerequisites - -Complete [Develop your app](develop.md). This topic requires a local Python -installation because the tools and Git hooks introduced here run on your -host. If you don't want to install Python locally, skip this topic. The same -checks run in CI in the [next topic](configure-github-actions.md). - -## Overview - -Linting, formatting, and type checking are automated ways to catch bugs, -enforce style, and spot type errors before code runs. Running them on every -commit, in CI, and in your editor catches problems early when they're cheap -to fix. - -In this section, you'll configure three tools for your Python application. -Ruff handles linting and formatting in a single fast pass. Pyright statically -checks your code for type errors. Pre-commit hooks run both of these -automatically before each Git commit so problems are caught locally before -they're committed. - -## Linting and formatting with Ruff - -Ruff is an extremely fast Python linter and formatter written in Rust. It replaces multiple tools like flake8, isort, and black with a single unified tool. - -Create a `pyproject.toml` file in your `python-docker-example` directory: - -{{< files name="python-docker-example" >}} - -{{< file path="pyproject.toml" status="new" >}} -```toml -# Configuration for code-quality tools. -# - [tool.ruff]: linting and formatting (https://docs.astral.sh/ruff/) -# - [tool.pyright]: static type checking (https://microsoft.github.io/pyright/) - -[tool.ruff] -target-version = "py312" - -[tool.ruff.lint] -select = [ - "E", # pycodestyle errors - "W", # pycodestyle warnings - "F", # pyflakes - "I", # isort - "B", # flake8-bugbear - "C4", # flake8-comprehensions - "UP", # pyupgrade - "ARG001", # unused arguments in functions -] -ignore = [ - "E501", # line too long, handled by black - "B008", # do not perform function calls in argument defaults - "W191", # indentation contains tabs - "B904", # Allow raising exceptions without from e, for HTTPException -] -``` -{{< /file >}} - -{{< /files >}} - -Install Ruff: - -```console -$ pip install ruff -``` - -If you're using a virtual environment, make sure it is activated so the `ruff` -command is available. - -Run these commands to check and format your code: - -```console -# Check for errors -$ ruff check . - -# Automatically fix fixable errors -$ ruff check --fix . - -# Format code -$ ruff format . -``` - -## Type checking with Pyright - -Pyright is a fast static type checker for Python that works well with modern Python features. - -Update `pyproject.toml` to add the Pyright configuration at the bottom. - -{{< files name="python-docker-example" >}} - -{{< file path="pyproject.toml" status="modified" hl_lines="25-29" >}} -```toml -# Configuration for code-quality tools. -# - [tool.ruff]: linting and formatting (https://docs.astral.sh/ruff/) -# - [tool.pyright]: static type checking (https://microsoft.github.io/pyright/) - -[tool.ruff] -target-version = "py312" - -[tool.ruff.lint] -select = [ - "E", # pycodestyle errors - "W", # pycodestyle warnings - "F", # pyflakes - "I", # isort - "B", # flake8-bugbear - "C4", # flake8-comprehensions - "UP", # pyupgrade - "ARG001", # unused arguments in functions -] -ignore = [ - "E501", # line too long, handled by black - "B008", # do not perform function calls in argument defaults - "W191", # indentation contains tabs - "B904", # Allow raising exceptions without from e, for HTTPException -] - -[tool.pyright] -typeCheckingMode = "strict" -pythonVersion = "3.12" -exclude = [".venv"] -``` -{{< /file >}} - -{{< /files >}} - -Install Pyright and run it: - -```console -$ pip install pyright -$ pyright -``` - -## Setting up pre-commit hooks - -Pre-commit hooks run checks automatically before each commit on your local -machine. Create a `.pre-commit-config.yaml` file in your `python-docker-example` -directory to set up Ruff hooks: - -{{< files name="python-docker-example" >}} - -{{< file path=".pre-commit-config.yaml" status="new" >}} -```yaml -# Pre-commit hook configuration. Runs Ruff (lint + format) on every -# `git commit`. See https://pre-commit.com/ - -repos: - - repo: https://github.com/astral-sh/ruff-pre-commit - rev: v0.15.15 - hooks: - - id: ruff - args: [--fix] - - id: ruff-format -``` -{{< /file >}} - -{{< /files >}} - -To install and use: - -```console -$ pip install pre-commit -$ pre-commit install -$ git commit -m "Test commit" # Automatically runs checks -``` - -## Summary - -In this section, you learned how to: - -- Configure and use Ruff for linting and formatting -- Set up Pyright for static type checking -- Automate checks with pre-commit hooks - -These tools help maintain code quality and catch errors early in development. - -Related information: - -- [Ruff documentation](https://docs.astral.sh/ruff/) -- [Pyright documentation](https://microsoft.github.io/pyright/) -- [pre-commit framework](https://pre-commit.com/) - -## Next steps - -- [Configure GitHub Actions](configure-github-actions.md) to run these checks automatically -- Customize linting rules to match your team's style preferences -- Explore advanced type checking features diff --git a/content/guides/python/secure-supply-chain.md b/content/guides/python/secure-supply-chain.md deleted file mode 100644 index 07c8b3f0ca60..000000000000 --- a/content/guides/python/secure-supply-chain.md +++ /dev/null @@ -1,144 +0,0 @@ ---- -title: Secure your Python image supply chain -linkTitle: Secure your supply chain -weight: 45 -keywords: python, sbom, provenance, attestations, docker scout, supply chain, security -description: Learn how to inspect, generate, and verify supply chain attestations for your Python container image. ---- - -## Prerequisites - -Complete [Configure CI/CD for your Python application](configure-github-actions.md). - -## Overview - -When you ship a container image, what's inside it and where it came from -matters. Supply chain attestations are signed records that answer questions -like which packages are in the image, what vulnerabilities affect them, how -the image was built, and what security checks it passed. - -In this section, you'll inspect the attestations that ship with your Docker -Hardened Image base, generate your own SBOM and provenance attestations -during CI, and pin the base image by digest so your builds are reproducible. - -The inspection commands in this topic are shown manually so you can see what -each one returns. In a real workflow you'd automate these checks with -[Docker Scout](/scout/), which runs the same scans on every push, -enforces policies in CI, and surfaces results in your registry and pull -requests. - -## Inspect the base image attestations - -Docker Hardened Images are built to SLSA Build Level 3 and ship with a set of -signed attestations covering bill-of-materials, vulnerabilities, build -provenance, and security scans. See -[DHI attestations](/manuals/dhi/core-concepts/attestations.md) for the full -list of types and how to verify their signatures with Cosign. - -List all the attestations available on the Python DHI: - -```console -$ docker scout attest list registry://dhi.io/python:3.12 -``` - -View the SBOM: - -```console -$ docker scout sbom registry://dhi.io/python:3.12 -``` - -Check known vulnerabilities: - -```console -$ docker scout cves registry://dhi.io/python:3.12 -``` - -> [!NOTE] -> -> The `registry://` prefix forces `docker scout` to fetch the image and its -> attestations from the registry instead of reading a locally pulled copy. If -> you've already pulled or built against the base image, the local copy -> doesn't have the attached attestations, so the prefix is required to see -> them. - -When you base your own image on a DHI image, these attestations stay attached to the base layer in the registry. Tools that inspect your image can follow the chain back to the DHI source. - -## Generate attestations for your image - -Update your GitHub Actions workflow to attach SBOM and provenance attestations to the image you push. - -Edit `.github/workflows/build.yml` and update the build-and-push step: - -```yaml {hl_lines="6-7"} -- name: Build and push Docker image - uses: docker/build-push-action@v6 - with: - context: . - push: true - sbom: true - provenance: mode=max - tags: ${{ steps.meta.outputs.tags }} -``` - -- `sbom: true` tells BuildKit to scan the built image and attach an SBOM attestation. -- `provenance: mode=max` records detailed build provenance, including the source repository, commit, and build parameters. - -The next time your workflow runs, the pushed image will carry these attestations alongside the image manifest in the registry. - -## Inspect your pushed image's attestations - -After your workflow pushes the image, inspect it the same way you inspected the base image: - -```console -$ docker scout attest list registry://DOCKER_USERNAME/REPO_NAME:latest -$ docker scout sbom registry://DOCKER_USERNAME/REPO_NAME:latest -``` - -The SBOM includes packages from every layer, including those inherited from `dhi.io/python:3.12`. The provenance record references the DHI base image by digest, so consumers of your image can trace the build chain back to the DHI source. - -## Pin the base image by digest - -Image tags like `dhi.io/python:3.12` move over time as new patches land. For reproducible builds, pin to an immutable digest. - -The Dockerfile uses two tags, `dhi.io/python:3.12-dev` in the builder stage -and `dhi.io/python:3.12` in the runtime stage. Each tag has its own digest, -so look up both: - -```console -$ docker buildx imagetools inspect dhi.io/python:3.12-dev --format "{{ .Manifest.Digest }}" -sha256:4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 -$ docker buildx imagetools inspect dhi.io/python:3.12 --format "{{ .Manifest.Digest }}" -sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 -``` - -Each digest is a 64-character hex string. Update your `Dockerfile` to reference -each digest on the matching `FROM` line: - -```dockerfile -FROM dhi.io/python:3.12-dev@sha256:4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945 AS builder -# ... -FROM dhi.io/python:3.12@sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 -``` - -> [!TIP] -> -> Pinning by digest also pins you to that image's vulnerabilities. Use [Dependabot](https://docs.github.com/en/code-security/dependabot) or [Renovate](https://docs.renovatebot.com/) to automate digest updates so you get a PR when a new patched image is available, with a changelog to review before merging. - -## Summary - -In this section, you learned how to: - -- Inspect the supply chain attestations that ship with the DHI base image, including SBOMs, CVE reports, VEX statements, and scan results -- Generate SBOM and provenance attestations for your own image in CI -- Pin base images by digest for reproducible builds - -Related information: - -- [DHI attestations](/manuals/dhi/core-concepts/attestations.md) -- [Verify a Docker Hardened Image](/manuals/dhi/how-to/verify.md) -- [Docker Scout](/scout/) -- [Build attestations](/manuals/build/metadata/attestations/_index.md) - -## Next steps - -In the next section, you'll deploy your application to Kubernetes. diff --git a/content/guides/r/_index.md b/content/guides/r/_index.md index c599db41a572..3c4e6c494750 100644 --- a/content/guides/r/_index.md +++ b/content/guides/r/_index.md @@ -5,16 +5,25 @@ description: Containerize R apps using Docker keywords: Docker, getting started, R, language summary: | This guide details how to containerize R applications using Docker. -toc_min: 1 -toc_max: 2 aliases: - /languages/r/ - /guides/languages/r/ -languages: [r] + - /language/R/build-images/ + - /language/R/run-containers/ + - /language/r/containerize/ + - /language/r/develop/ + - /language/r/configure-ci-cd/ + - /language/r/deploy/ + - /guides/r/configure-ci-cd/ + - /guides/r/containerize/ + - /guides/r/deploy/ + - /guides/r/develop/ params: + tags: [cicd] time: 10 minutes --- + The R language-specific guide teaches you how to containerize a R application using Docker. In this guide, you’ll learn how to: - Containerize and run a R application @@ -23,3 +32,570 @@ The R language-specific guide teaches you how to containerize a R application us - Deploy your containerized R application locally to Kubernetes to test and debug your deployment Start by containerizing an existing R application. + +## Containerize a R application + +### Prerequisites + +- You have a [git client](https://git-scm.com/downloads). The examples in this section use a command-line based git client, but you can use any client. + +### Overview + +This section walks you through containerizing and running a R application. + +### Get the sample application + +The sample application uses the popular [Shiny](https://shiny.posit.co/) framework. + +Clone the sample application to use with this guide. Open a terminal, change directory to a directory that you want to work in, and run the following command to clone the repository: + +```console +$ git clone https://github.com/mfranzon/r-docker-dev.git && cd r-docker-dev +``` + +You should now have the following contents in your `r-docker-dev` +directory. + +```text +├── r-docker-dev/ +│ ├── src/ +│ │ └── app.R +│ ├── src_db/ +│ │ └── app_db.R +│ ├── compose.yaml +│ ├── Dockerfile +│ └── README.md +``` + +To learn more about the files in the repository, see the following: + +- [Dockerfile](/reference/dockerfile.md) +- [.dockerignore](/reference/dockerfile.md#dockerignore-file) +- [compose.yaml](/reference/compose-file/_index.md) + +### Run the application + +Inside the `r-docker-dev` directory, run the following command in a +terminal. + +```console +$ docker compose up --build +``` + +Open a browser and view the application at [http://localhost:3838](http://localhost:3838). You should see a simple Shiny application. + +In the terminal, press `ctrl`+`c` to stop the application. + +#### Run the application in the background + +You can run the application detached from the terminal by adding the `-d` +option. Inside the `r-docker-dev` directory, run the following command +in a terminal. + +```console +$ docker compose up --build -d +``` + +Open a browser and view the application at [http://localhost:3838](http://localhost:3838). + +You should see a simple Shiny application. + +In the terminal, run the following command to stop the application. + +```console +$ docker compose down +``` + +For more information about Compose commands, see the [Compose CLI +reference](/reference/cli/docker/compose/). + +### Summary + +In this section, you learned how you can containerize and run your R +application using Docker. + +Related information: + +- [Docker Compose overview](/manuals/compose/_index.md) + +### Next steps + +In the next section, you'll learn how you can develop your application using +containers. + +## Use containers for R development + +### Prerequisites + +Complete [Containerize a R application](containerize.md). + +### Overview + +In this section, you'll learn how to set up a development environment for your containerized application. This includes: + +- Adding a local database and persisting data +- Configuring Compose to automatically update your running Compose services as you edit and save your code + +### Get the sample application + +You'll need to clone a new repository to get a sample application that includes logic to connect to the database. + +Change to a directory where you want to clone the repository and run the following command. + +```console +$ git clone https://github.com/mfranzon/r-docker-dev.git +``` + +### Configure the application to use the database + +To try the connection between the Shiny application and the local database you have to modify the `Dockerfile` changing the `COPY` instruction: + +```diff +-COPY src/ . ++COPY src_db/ . +``` + +### Add a local database and persist data + +You can use containers to set up local services, like a database. In this section, you'll update the `compose.yaml` file to define a database service and a volume to persist data. + +In the cloned repository's directory, open the `compose.yaml` file in an IDE or text editor. + +In the `compose.yaml` file, you need to un-comment the properties for configuring the database. You must also mount the database password file and set an environment variable on the `shiny-app` service pointing to the location of the file in the container. + +The following is the updated `compose.yaml` file. + +```yaml +services: + shiny-app: + build: + context: . + dockerfile: Dockerfile + ports: + - 3838:3838 + environment: + - POSTGRES_PASSWORD_FILE=/run/secrets/db-password + depends_on: + db: + condition: service_healthy + secrets: + - db-password + db: + image: postgres:18 + restart: always + user: postgres + secrets: + - db-password + volumes: + - db-data:/var/lib/postgresql + environment: + - POSTGRES_DB=example + - POSTGRES_PASSWORD_FILE=/run/secrets/db-password + expose: + - 5432 + healthcheck: + test: ["CMD", "pg_isready"] + interval: 10s + timeout: 5s + retries: 5 +volumes: + db-data: +secrets: + db-password: + file: db/password.txt +``` + +> [!NOTE] +> +> To learn more about the instructions in the Compose file, see [Compose file +> reference](/reference/compose-file/). + +Before you run the application using Compose, notice that this Compose file specifies a `password.txt` file to hold the database's password. You must create this file as it's not included in the source repository. + +In the cloned repository's directory, create a new directory named `db` and inside that directory create a file named `password.txt` that contains the password for the database. Using your favorite IDE or text editor, add the following contents to the `password.txt` file. + +```text +mysecretpassword +``` + +Save and close the `password.txt` file. + +You should now have the following contents in your `r-docker-dev` +directory. + +```text +├── r-docker-dev/ +│ ├── db/ +│ │ └── password.txt +│ ├── src/ +│ │ └── app.R +│ ├── src_db/ +│ │ └── app_db.R +│ ├── requirements.txt +│ ├── .dockerignore +│ ├── compose.yaml +│ ├── Dockerfile +│ └── README.md +``` + +Now, run the following `docker compose up` command to start your application. + +```console +$ docker compose up --build +``` + +Now test your DB connection opening a browser at: + +```console +http://localhost:3838 +``` + +You should see a pop-up message: + +```text +DB CONNECTED +``` + +Press `ctrl+c` in the terminal to stop your application. + +### Automatically update services + +Use Compose Watch to automatically update your running Compose services as you +edit and save your code. For more details about Compose Watch, see [Use Compose +Watch](/manuals/compose/how-tos/file-watch.md). + +Lines 15 to 18 in the `compose.yaml` file contain properties that trigger Docker +to rebuild the image when a file in the current working directory is changed: + +```yaml {hl_lines="15-18",linenos=true} +services: + shiny-app: + build: + context: . + dockerfile: Dockerfile + ports: + - 3838:3838 + environment: + - POSTGRES_PASSWORD_FILE=/run/secrets/db-password + depends_on: + db: + condition: service_healthy + secrets: + - db-password + develop: + watch: + - action: rebuild + path: . + db: + image: postgres:18 + restart: always + user: postgres + secrets: + - db-password + volumes: + - db-data:/var/lib/postgresql + environment: + - POSTGRES_DB=example + - POSTGRES_PASSWORD_FILE=/run/secrets/db-password + expose: + - 5432 + healthcheck: + test: ["CMD", "pg_isready"] + interval: 10s + timeout: 5s + retries: 5 +volumes: + db-data: +secrets: + db-password: + file: db/password.txt +``` + +Run the following command to run your application with Compose Watch. + +```console +$ docker compose watch +``` + +Now, if you modify your `app.R` you will see the changes in real time without re-building the image! + +Press `ctrl+c` in the terminal to stop your application. + +### Summary + +In this section, you took a look at setting up your Compose file to add a local +database and persist data. You also learned how to use Compose Watch to automatically rebuild and run your container when you update your code. + +Related information: + +- [Compose file reference](/reference/compose-file/) +- [Compose file watch](/manuals/compose/how-tos/file-watch.md) +- [Multi-stage builds](/manuals/build/building/multi-stage.md) + +### Next steps + +In the next section, you'll take a look at how to set up a CI/CD pipeline using GitHub Actions. + +## Configure CI/CD for your R application + +### Prerequisites + +Complete all the previous sections of this guide, starting with [Containerize a R application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. + +### Overview + +In this section, you'll learn how to set up and use GitHub Actions to build and test your Docker image as well as push it to Docker Hub. You will complete the following steps: + +1. Create a new repository on GitHub. +2. Define the GitHub Actions workflow. +3. Run the workflow. + +### Step one: Create the repository + +Create a GitHub repository, configure the Docker Hub credentials, and push your source code. + +1. [Create a new repository](https://github.com/new) on GitHub. + +2. Open the repository **Settings**, and go to **Secrets and variables** > + **Actions**. + +3. Create a new **Repository variable** named `DOCKER_USERNAME` and your Docker ID as a value. + +4. Create a new [Personal Access Token (PAT)](/manuals/security/access-tokens.md#create-an-access-token) for Docker Hub. You can name this token `docker-tutorial`. Make sure access permissions include Read and Write. + +5. Add the PAT as a **Repository secret** in your GitHub repository, with the name + `DOCKERHUB_TOKEN`. + +6. In your local repository on your machine, run the following command to change + the origin to the repository you just created. Make sure you change + `your-username` to your GitHub username and `your-repository` to the name of + the repository you created. + + ```console + $ git remote set-url origin https://github.com/your-username/your-repository.git + ``` + +7. Run the following commands to stage, commit, and push your local repository to GitHub. + + ```console + $ git add -A + $ git commit -m "my commit" + $ git push -u origin main + ``` + +### Step two: Set up the workflow + +Set up your GitHub Actions workflow for building, testing, and pushing the image +to Docker Hub. + +1. Go to your repository on GitHub and then select the **Actions** tab. + +2. Select **set up a workflow yourself**. + + This takes you to a page for creating a new GitHub actions workflow file in + your repository, under `.github/workflows/main.yml` by default. + +3. In the editor window, copy and paste the following YAML configuration. + + ```yaml + name: ci + + on: + push: + branches: + - main + + jobs: + build: + runs-on: ubuntu-latest + steps: + - name: Login to Docker Hub + uses: docker/login-action@{{% param "login_action_version" %}} + with: + username: ${{ vars.DOCKER_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} + + - name: Build and push + uses: docker/build-push-action@{{% param "build_push_action_version" %}} + with: + platforms: linux/amd64,linux/arm64 + push: true + tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest + ``` + + For more information about the YAML syntax for `docker/build-push-action`, + refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). + +### Step three: Run the workflow + +Save the workflow file and run the job. + +1. Select **Commit changes...** and push the changes to the `main` branch. + + After pushing the commit, the workflow starts automatically. + +2. Go to the **Actions** tab. It displays the workflow. + + Selecting the workflow shows you the breakdown of all the steps. + +3. When the workflow is complete, go to your + [repositories on Docker Hub](https://hub.docker.com/repositories). + + If you see the new repository in that list, it means the GitHub Actions + successfully pushed the image to Docker Hub. + +### Summary + +In this section, you learned how to set up a GitHub Actions workflow for your R application. + +Related information: + +- [Introduction to GitHub Actions](/guides/gha.md) +- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) +- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) + +### Next steps + +Next, learn how you can locally test and debug your workloads on Kubernetes before deploying. + +## Test your R deployment + +### Prerequisites + +- Complete all the previous sections of this guide, starting with [Containerize a R application](containerize.md). +- [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. + +### Overview + +In this section, you'll learn how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. This allows you to test and debug your workloads on Kubernetes locally before deploying. + +### Create a Kubernetes YAML file + +In your `r-docker-dev` directory, create a file named +`docker-r-kubernetes.yaml`. Open the file in an IDE or text editor and add +the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker +username and the name of the repository that you created in [Configure CI/CD for +your R application](configure-ci-cd.md). + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: docker-r-demo + namespace: default +spec: + replicas: 1 + selector: + matchLabels: + service: shiny + template: + metadata: + labels: + service: shiny + spec: + containers: + - name: shiny-service + image: DOCKER_USERNAME/REPO_NAME + imagePullPolicy: Always + env: + - name: POSTGRES_PASSWORD + value: mysecretpassword +--- +apiVersion: v1 +kind: Service +metadata: + name: service-entrypoint + namespace: default +spec: + type: NodePort + selector: + service: shiny + ports: + - port: 3838 + targetPort: 3838 + nodePort: 30001 +``` + +In this Kubernetes YAML file, there are two objects, separated by the `---`: + +- A Deployment, describing a scalable group of identical pods. In this case, + you'll get just one replica, or copy of your pod. That pod, which is + described under `template`, has just one container in it. The + container is created from the image built by GitHub Actions in [Configure CI/CD for + your R application](configure-ci-cd.md). +- A NodePort service, which will route traffic from port 30001 on your host to + port 3838 inside the pods it routes to, allowing you to reach your app + from the network. + +To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). + +### Deploy and check your application + +1. In a terminal, navigate to `r-docker-dev` and deploy your application to + Kubernetes. + + ```console + $ kubectl apply -f docker-r-kubernetes.yaml + ``` + + You should see output that looks like the following, indicating your Kubernetes objects were created successfully. + + ```text + deployment.apps/docker-r-demo created + service/service-entrypoint created + ``` + +2. Make sure everything worked by listing your deployments. + + ```console + $ kubectl get deployments + ``` + + Your deployment should be listed as follows: + + ```shell + NAME READY UP-TO-DATE AVAILABLE AGE + docker-r-demo 1/1 1 1 15s + ``` + + This indicates all one of the pods you asked for in your YAML are up and running. Do the same check for your services. + + ```console + $ kubectl get services + ``` + + You should get output like the following. + + ```shell + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + kubernetes ClusterIP 10.96.0.1 443/TCP 23h + service-entrypoint NodePort 10.99.128.230 3838:30001/TCP 75s + ``` + + In addition to the default `kubernetes` service, you can see your `service-entrypoint` service, accepting traffic on port 30001/TCP. + +3. In a browser, visit the following address. Note that a database was not deployed in + this example. + + ```console + http://localhost:30001/ + ``` + +4. Run the following command to tear down your application. + + ```console + $ kubectl delete -f docker-r-kubernetes.yaml + ``` + +### Summary + +In this section, you learned how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. + +Related information: + +- [Kubernetes documentation](https://kubernetes.io/docs/home/) +- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) +- [Swarm mode overview](/manuals/engine/swarm/_index.md) diff --git a/content/guides/r/configure-ci-cd.md b/content/guides/r/configure-ci-cd.md deleted file mode 100644 index 662a2155e14b..000000000000 --- a/content/guides/r/configure-ci-cd.md +++ /dev/null @@ -1,133 +0,0 @@ ---- -title: Configure CI/CD for your R application -linkTitle: Configure CI/CD -weight: 40 -keywords: ci/cd, github actions, R, shiny -description: Learn how to configure CI/CD using GitHub Actions for your R application. -aliases: - - /language/r/configure-ci-cd/ - - /guides/language/r/configure-ci-cd/ ---- - -## Prerequisites - -Complete all the previous sections of this guide, starting with [Containerize a R application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. - -## Overview - -In this section, you'll learn how to set up and use GitHub Actions to build and test your Docker image as well as push it to Docker Hub. You will complete the following steps: - -1. Create a new repository on GitHub. -2. Define the GitHub Actions workflow. -3. Run the workflow. - -## Step one: Create the repository - -Create a GitHub repository, configure the Docker Hub credentials, and push your source code. - -1. [Create a new repository](https://github.com/new) on GitHub. - -2. Open the repository **Settings**, and go to **Secrets and variables** > - **Actions**. - -3. Create a new **Repository variable** named `DOCKER_USERNAME` and your Docker ID as a value. - -4. Create a new [Personal Access Token (PAT)](/manuals/security/access-tokens.md#create-an-access-token) for Docker Hub. You can name this token `docker-tutorial`. Make sure access permissions include Read and Write. - -5. Add the PAT as a **Repository secret** in your GitHub repository, with the name - `DOCKERHUB_TOKEN`. - -6. In your local repository on your machine, run the following command to change - the origin to the repository you just created. Make sure you change - `your-username` to your GitHub username and `your-repository` to the name of - the repository you created. - - ```console - $ git remote set-url origin https://github.com/your-username/your-repository.git - ``` - -7. Run the following commands to stage, commit, and push your local repository to GitHub. - - ```console - $ git add -A - $ git commit -m "my commit" - $ git push -u origin main - ``` - -## Step two: Set up the workflow - -Set up your GitHub Actions workflow for building, testing, and pushing the image -to Docker Hub. - -1. Go to your repository on GitHub and then select the **Actions** tab. - -2. Select **set up a workflow yourself**. - - This takes you to a page for creating a new GitHub actions workflow file in - your repository, under `.github/workflows/main.yml` by default. - -3. In the editor window, copy and paste the following YAML configuration. - - ```yaml - name: ci - - on: - push: - branches: - - main - - jobs: - build: - runs-on: ubuntu-latest - steps: - - name: Login to Docker Hub - uses: docker/login-action@{{% param "login_action_version" %}} - with: - username: ${{ vars.DOCKER_USERNAME }} - password: ${{ secrets.DOCKERHUB_TOKEN }} - - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} - - - name: Build and push - uses: docker/build-push-action@{{% param "build_push_action_version" %}} - with: - platforms: linux/amd64,linux/arm64 - push: true - tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest - ``` - - For more information about the YAML syntax for `docker/build-push-action`, - refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). - -## Step three: Run the workflow - -Save the workflow file and run the job. - -1. Select **Commit changes...** and push the changes to the `main` branch. - - After pushing the commit, the workflow starts automatically. - -2. Go to the **Actions** tab. It displays the workflow. - - Selecting the workflow shows you the breakdown of all the steps. - -3. When the workflow is complete, go to your - [repositories on Docker Hub](https://hub.docker.com/repositories). - - If you see the new repository in that list, it means the GitHub Actions - successfully pushed the image to Docker Hub. - -## Summary - -In this section, you learned how to set up a GitHub Actions workflow for your R application. - -Related information: - -- [Introduction to GitHub Actions](/guides/gha.md) -- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) -- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) - -## Next steps - -Next, learn how you can locally test and debug your workloads on Kubernetes before deploying. diff --git a/content/guides/r/containerize.md b/content/guides/r/containerize.md deleted file mode 100644 index fd8c966f9f30..000000000000 --- a/content/guides/r/containerize.md +++ /dev/null @@ -1,100 +0,0 @@ ---- -title: Containerize a R application -linkTitle: Containerize your app -weight: 10 -keywords: R, containerize, initialize -description: Learn how to containerize a R application. -aliases: - - /language/R/build-images/ - - /language/R/run-containers/ - - /language/r/containerize/ - - /guides/language/r/containerize/ ---- - -## Prerequisites - -- You have a [git client](https://git-scm.com/downloads). The examples in this section use a command-line based git client, but you can use any client. - -## Overview - -This section walks you through containerizing and running a R application. - -## Get the sample application - -The sample application uses the popular [Shiny](https://shiny.posit.co/) framework. - -Clone the sample application to use with this guide. Open a terminal, change directory to a directory that you want to work in, and run the following command to clone the repository: - -```console -$ git clone https://github.com/mfranzon/r-docker-dev.git && cd r-docker-dev -``` - -You should now have the following contents in your `r-docker-dev` -directory. - -```text -├── r-docker-dev/ -│ ├── src/ -│ │ └── app.R -│ ├── src_db/ -│ │ └── app_db.R -│ ├── compose.yaml -│ ├── Dockerfile -│ └── README.md -``` - -To learn more about the files in the repository, see the following: - -- [Dockerfile](/reference/dockerfile.md) -- [.dockerignore](/reference/dockerfile.md#dockerignore-file) -- [compose.yaml](/reference/compose-file/_index.md) - -## Run the application - -Inside the `r-docker-dev` directory, run the following command in a -terminal. - -```console -$ docker compose up --build -``` - -Open a browser and view the application at [http://localhost:3838](http://localhost:3838). You should see a simple Shiny application. - -In the terminal, press `ctrl`+`c` to stop the application. - -### Run the application in the background - -You can run the application detached from the terminal by adding the `-d` -option. Inside the `r-docker-dev` directory, run the following command -in a terminal. - -```console -$ docker compose up --build -d -``` - -Open a browser and view the application at [http://localhost:3838](http://localhost:3838). - -You should see a simple Shiny application. - -In the terminal, run the following command to stop the application. - -```console -$ docker compose down -``` - -For more information about Compose commands, see the [Compose CLI -reference](/reference/cli/docker/compose/). - -## Summary - -In this section, you learned how you can containerize and run your R -application using Docker. - -Related information: - -- [Docker Compose overview](/manuals/compose/_index.md) - -## Next steps - -In the next section, you'll learn how you can develop your application using -containers. diff --git a/content/guides/r/deploy.md b/content/guides/r/deploy.md deleted file mode 100644 index 70c10b216faa..000000000000 --- a/content/guides/r/deploy.md +++ /dev/null @@ -1,147 +0,0 @@ ---- -title: Test your R deployment -linkTitle: Test your deployment -weight: 50 -keywords: deploy, kubernetes, R -description: Learn how to develop locally using Kubernetes -aliases: - - /language/r/deploy/ - - /guides/language/r/deploy/ ---- - -## Prerequisites - -- Complete all the previous sections of this guide, starting with [Containerize a R application](containerize.md). -- [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. - -## Overview - -In this section, you'll learn how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. This allows you to test and debug your workloads on Kubernetes locally before deploying. - -## Create a Kubernetes YAML file - -In your `r-docker-dev` directory, create a file named -`docker-r-kubernetes.yaml`. Open the file in an IDE or text editor and add -the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker -username and the name of the repository that you created in [Configure CI/CD for -your R application](configure-ci-cd.md). - -```yaml -apiVersion: apps/v1 -kind: Deployment -metadata: - name: docker-r-demo - namespace: default -spec: - replicas: 1 - selector: - matchLabels: - service: shiny - template: - metadata: - labels: - service: shiny - spec: - containers: - - name: shiny-service - image: DOCKER_USERNAME/REPO_NAME - imagePullPolicy: Always - env: - - name: POSTGRES_PASSWORD - value: mysecretpassword ---- -apiVersion: v1 -kind: Service -metadata: - name: service-entrypoint - namespace: default -spec: - type: NodePort - selector: - service: shiny - ports: - - port: 3838 - targetPort: 3838 - nodePort: 30001 -``` - -In this Kubernetes YAML file, there are two objects, separated by the `---`: - -- A Deployment, describing a scalable group of identical pods. In this case, - you'll get just one replica, or copy of your pod. That pod, which is - described under `template`, has just one container in it. The - container is created from the image built by GitHub Actions in [Configure CI/CD for - your R application](configure-ci-cd.md). -- A NodePort service, which will route traffic from port 30001 on your host to - port 3838 inside the pods it routes to, allowing you to reach your app - from the network. - -To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). - -## Deploy and check your application - -1. In a terminal, navigate to `r-docker-dev` and deploy your application to - Kubernetes. - - ```console - $ kubectl apply -f docker-r-kubernetes.yaml - ``` - - You should see output that looks like the following, indicating your Kubernetes objects were created successfully. - - ```text - deployment.apps/docker-r-demo created - service/service-entrypoint created - ``` - -2. Make sure everything worked by listing your deployments. - - ```console - $ kubectl get deployments - ``` - - Your deployment should be listed as follows: - - ```shell - NAME READY UP-TO-DATE AVAILABLE AGE - docker-r-demo 1/1 1 1 15s - ``` - - This indicates all one of the pods you asked for in your YAML are up and running. Do the same check for your services. - - ```console - $ kubectl get services - ``` - - You should get output like the following. - - ```shell - NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE - kubernetes ClusterIP 10.96.0.1 443/TCP 23h - service-entrypoint NodePort 10.99.128.230 3838:30001/TCP 75s - ``` - - In addition to the default `kubernetes` service, you can see your `service-entrypoint` service, accepting traffic on port 30001/TCP. - -3. In a browser, visit the following address. Note that a database was not deployed in - this example. - - ```console - http://localhost:30001/ - ``` - -4. Run the following command to tear down your application. - - ```console - $ kubectl delete -f docker-r-kubernetes.yaml - ``` - -## Summary - -In this section, you learned how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. - -Related information: - -- [Kubernetes documentation](https://kubernetes.io/docs/home/) -- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) -- [Swarm mode overview](/manuals/engine/swarm/_index.md) diff --git a/content/guides/r/develop.md b/content/guides/r/develop.md deleted file mode 100644 index 0427464e298a..000000000000 --- a/content/guides/r/develop.md +++ /dev/null @@ -1,221 +0,0 @@ ---- -title: Use containers for R development -linkTitle: Develop your app -weight: 20 -keywords: R, local, development -description: Learn how to develop your R application locally. -aliases: - - /language/r/develop/ - - /guides/language/r/develop/ ---- - -## Prerequisites - -Complete [Containerize a R application](containerize.md). - -## Overview - -In this section, you'll learn how to set up a development environment for your containerized application. This includes: - -- Adding a local database and persisting data -- Configuring Compose to automatically update your running Compose services as you edit and save your code - -## Get the sample application - -You'll need to clone a new repository to get a sample application that includes logic to connect to the database. - -Change to a directory where you want to clone the repository and run the following command. - -```console -$ git clone https://github.com/mfranzon/r-docker-dev.git -``` - -## Configure the application to use the database - -To try the connection between the Shiny application and the local database you have to modify the `Dockerfile` changing the `COPY` instruction: - -```diff --COPY src/ . -+COPY src_db/ . -``` - -## Add a local database and persist data - -You can use containers to set up local services, like a database. In this section, you'll update the `compose.yaml` file to define a database service and a volume to persist data. - -In the cloned repository's directory, open the `compose.yaml` file in an IDE or text editor. - -In the `compose.yaml` file, you need to un-comment the properties for configuring the database. You must also mount the database password file and set an environment variable on the `shiny-app` service pointing to the location of the file in the container. - -The following is the updated `compose.yaml` file. - -```yaml -services: - shiny-app: - build: - context: . - dockerfile: Dockerfile - ports: - - 3838:3838 - environment: - - POSTGRES_PASSWORD_FILE=/run/secrets/db-password - depends_on: - db: - condition: service_healthy - secrets: - - db-password - db: - image: postgres:18 - restart: always - user: postgres - secrets: - - db-password - volumes: - - db-data:/var/lib/postgresql - environment: - - POSTGRES_DB=example - - POSTGRES_PASSWORD_FILE=/run/secrets/db-password - expose: - - 5432 - healthcheck: - test: ["CMD", "pg_isready"] - interval: 10s - timeout: 5s - retries: 5 -volumes: - db-data: -secrets: - db-password: - file: db/password.txt -``` - -> [!NOTE] -> -> To learn more about the instructions in the Compose file, see [Compose file -> reference](/reference/compose-file/). - -Before you run the application using Compose, notice that this Compose file specifies a `password.txt` file to hold the database's password. You must create this file as it's not included in the source repository. - -In the cloned repository's directory, create a new directory named `db` and inside that directory create a file named `password.txt` that contains the password for the database. Using your favorite IDE or text editor, add the following contents to the `password.txt` file. - -```text -mysecretpassword -``` - -Save and close the `password.txt` file. - -You should now have the following contents in your `r-docker-dev` -directory. - -```text -├── r-docker-dev/ -│ ├── db/ -│ │ └── password.txt -│ ├── src/ -│ │ └── app.R -│ ├── src_db/ -│ │ └── app_db.R -│ ├── requirements.txt -│ ├── .dockerignore -│ ├── compose.yaml -│ ├── Dockerfile -│ └── README.md -``` - -Now, run the following `docker compose up` command to start your application. - -```console -$ docker compose up --build -``` - -Now test your DB connection opening a browser at: - -```console -http://localhost:3838 -``` - -You should see a pop-up message: - -```text -DB CONNECTED -``` - -Press `ctrl+c` in the terminal to stop your application. - -## Automatically update services - -Use Compose Watch to automatically update your running Compose services as you -edit and save your code. For more details about Compose Watch, see [Use Compose -Watch](/manuals/compose/how-tos/file-watch.md). - -Lines 15 to 18 in the `compose.yaml` file contain properties that trigger Docker -to rebuild the image when a file in the current working directory is changed: - -```yaml {hl_lines="15-18",linenos=true} -services: - shiny-app: - build: - context: . - dockerfile: Dockerfile - ports: - - 3838:3838 - environment: - - POSTGRES_PASSWORD_FILE=/run/secrets/db-password - depends_on: - db: - condition: service_healthy - secrets: - - db-password - develop: - watch: - - action: rebuild - path: . - db: - image: postgres:18 - restart: always - user: postgres - secrets: - - db-password - volumes: - - db-data:/var/lib/postgresql - environment: - - POSTGRES_DB=example - - POSTGRES_PASSWORD_FILE=/run/secrets/db-password - expose: - - 5432 - healthcheck: - test: ["CMD", "pg_isready"] - interval: 10s - timeout: 5s - retries: 5 -volumes: - db-data: -secrets: - db-password: - file: db/password.txt -``` - -Run the following command to run your application with Compose Watch. - -```console -$ docker compose watch -``` - -Now, if you modify your `app.R` you will see the changes in real time without re-building the image! - -Press `ctrl+c` in the terminal to stop your application. - -## Summary - -In this section, you took a look at setting up your Compose file to add a local -database and persist data. You also learned how to use Compose Watch to automatically rebuild and run your container when you update your code. - -Related information: - -- [Compose file reference](/reference/compose-file/) -- [Compose file watch](/manuals/compose/how-tos/file-watch.md) -- [Multi-stage builds](/manuals/build/building/multi-stage.md) - -## Next steps - -In the next section, you'll take a look at how to set up a CI/CD pipeline using GitHub Actions. diff --git a/content/guides/rag-ollama/_index.md b/content/guides/rag-ollama/_index.md index 79bfff446fbc..2bd3e2e325bf 100644 --- a/content/guides/rag-ollama/_index.md +++ b/content/guides/rag-ollama/_index.md @@ -6,16 +6,277 @@ linkTitle: RAG Ollama application summary: | This guide demonstrates how to use Docker to deploy Retrieval-Augmented Generation (RAG) models with Ollama. -tags: [ai] aliases: - /guides/use-case/rag-ollama/ + - /guides/rag-ollama/containerize/ + - /guides/rag-ollama/develop/ params: + tags: [ai] time: 20 minutes --- + The Retrieval Augmented Generation (RAG) guide teaches you how to containerize an existing RAG application using Docker. The example application is a RAG that acts like a sommelier, giving you the best pairings between wines and food. In this guide, you’ll learn how to: - Containerize and run a RAG application - Set up a local environment to run the complete RAG stack locally for development Start by containerizing an existing RAG application. + +## Containerize a RAG application + +### Overview + +This section walks you through containerizing a RAG application using Docker. + +> [!NOTE] +> You can see more samples of containerized GenAI applications in the [GenAI Stack](https://github.com/docker/genai-stack) demo applications. + +### Get the sample application + +The sample application used in this guide is an example of RAG application, made by three main components, which are the building blocks for every RAG application. A Large Language Model hosted somewhere, in this case it is hosted in a container and served via [Ollama](https://ollama.ai/). A vector database, [Qdrant](https://qdrant.tech/), to store the embeddings of local data, and a web application, using [Streamlit](https://streamlit.io/) to offer the best user experience to the user. + +Clone the sample application. Open a terminal, change directory to a directory that you want to work in, and run the following command to clone the repository: + +```console +$ git clone https://github.com/mfranzon/winy.git +``` + +You should now have the following files in your `winy` directory. + +```text +├── winy/ +│ ├── .gitignore +│ ├── app/ +│ │ ├── main.py +│ │ ├── Dockerfile +| | └── requirements.txt +│ ├── tools/ +│ │ ├── create_db.py +│ │ ├── create_embeddings.py +│ │ ├── requirements.txt +│ │ ├── test.py +| | └── download_model.sh +│ ├── docker-compose.yaml +│ ├── wine_database.db +│ ├── LICENSE +│ └── README.md +``` + +### Containerizing your application: Essentials + +Containerizing an application involves packaging it along with its dependencies into a container, which ensures consistency across different environments. Here’s what you need to containerize an app like Winy : + +1. Dockerfile: A Dockerfile that contains instructions on how to build a Docker image for your application. It specifies the base image, dependencies, configuration files, and the command to run your application. + +2. Docker Compose File: Docker Compose is a tool for defining and running multi-container Docker applications. A Compose file allows you to configure your application's services, networks, and volumes in a single file. + +### Run the application + +Inside the `winy` directory, run the following command in a +terminal. + +```console +$ docker compose up --build +``` + +Docker builds and runs your application. Depending on your network connection, it may take several minutes to download all the dependencies. You'll see a message like the following in the terminal when the application is running. + +```console +server-1 | You can now view your Streamlit app in your browser. +server-1 | +server-1 | URL: http://0.0.0.0:8501 +server-1 | +``` + +Open a browser and view the application at [http://localhost:8501](http://localhost:8501). You should see a simple Streamlit application. + +The application requires a Qdrant database service and an LLM service to work properly. If you have access to services that you ran outside of Docker, specify the connection information in the `docker-compose.yaml`. + +```yaml +winy: + build: + context: ./app + dockerfile: Dockerfile + environment: + - QDRANT_CLIENT=http://qdrant:6333 # Specifies the url for the qdrant database + - OLLAMA=http://ollama:11434 # Specifies the url for the ollama service + container_name: winy + ports: + - "8501:8501" + depends_on: + - qdrant + - ollama +``` + +If you don't have the services running, continue with this guide to learn how you can run some or all of these services with Docker. +Remember that the `ollama` service is empty; it doesn't have any model. For this reason you need to pull a model before starting to use the RAG application. All the instructions are in the following page. + +In the terminal, press `ctrl`+`c` to stop the application. + +### Summary + +In this section, you learned how you can containerize and run your RAG +application using Docker. + +### Next steps + +In the next section, you'll learn how to properly configure the application with your preferred LLM model, completely locally, using Docker. + +## Use containers for RAG development + +### Prerequisites + +Complete [Containerize a RAG application](containerize.md). + +### Overview + +In this section, you'll learn how to set up a development environment to access all the services that your generative RAG application needs. This includes: + +- Adding a local database +- Adding a local or remote LLM service + +> [!NOTE] +> You can see more samples of containerized GenAI applications in the [GenAI Stack](https://github.com/docker/genai-stack) demo applications. + +### Add a local database + +You can use containers to set up local services, like a database. In this section, you'll explore the database service in the `docker-compose.yaml` file. + +To run the database service: + +1. In the cloned repository's directory, open the `docker-compose.yaml` file in an IDE or text editor. + +2. In the `docker-compose.yaml` file, you'll see the following: + + ```yaml + services: + qdrant: + image: qdrant/qdrant + container_name: qdrant + ports: + - "6333:6333" + volumes: + - qdrant_data:/qdrant/storage + ``` + + > [!NOTE] + > To learn more about Qdrant, see the [Qdrant Official Docker Image](https://hub.docker.com/r/qdrant/qdrant). + +3. Start the application. Inside the `winy` directory, run the following command in a terminal. + + ```console + $ docker compose up --build + ``` + +4. Access the application. Open a browser and view the application at [http://localhost:8501](http://localhost:8501). You should see a simple Streamlit application. + +5. Stop the application. In the terminal, press `ctrl`+`c` to stop the application. + +### Add a local or remote LLM service + +The sample application supports both [Ollama](https://ollama.ai/). This guide provides instructions for the following scenarios: + +- Run Ollama in a container +- Run Ollama outside of a container + +While all platforms can use any of the previous scenarios, the performance and +GPU support may vary. You can use the following guidelines to help you choose the appropriate option: + +- Run Ollama in a container if you're on Linux, and using a native installation of the Docker Engine, or Windows 10/11, and using Docker Desktop, you + have a CUDA-supported GPU, and your system has at least 8 GB of RAM. +- Run Ollama outside of a container if running Docker Desktop on a Linux Machine. + +Choose one of the following options for your LLM service. + +{{< tabs >}} +{{< tab name="Run Ollama in a container" >}} + +When running Ollama in a container, you should have a CUDA-supported GPU. While you can run Ollama in a container without a supported GPU, the performance may not be acceptable. Only Linux and Windows 11 support GPU access to containers. + +To run Ollama in a container and provide GPU access: + +1. Install the prerequisites. + - For Docker Engine on Linux, install the [NVIDIA Container Toolkilt](https://github.com/NVIDIA/nvidia-container-toolkit). + - For Docker Desktop on Windows 10/11, install the latest [NVIDIA driver](https://www.nvidia.com/Download/index.aspx) and make sure you are using the [WSL2 backend](/manuals/desktop/features/wsl/_index.md#turn-on-docker-desktop-wsl-2) +2. The `docker-compose.yaml` file already contains the necessary instructions. In your own apps, you'll need to add the Ollama service in your `docker-compose.yaml`. The following is + the updated `docker-compose.yaml`: + + ```yaml + ollama: + image: ollama/ollama + container_name: ollama + ports: + - "8000:8000" + deploy: + resources: + reservations: + devices: + - driver: nvidia + count: 1 + capabilities: [gpu] + ``` + + > [!NOTE] + > For more details about the Compose instructions, see [Turn on GPU access with Docker Compose](/manuals/compose/how-tos/gpu-support.md). + +3. Once the Ollama container is up and running it is possible to use the `download_model.sh` inside the `tools` folder with this command: + + ```console + . ./download_model.sh + ``` + +Pulling an Ollama model can take several minutes. + +{{< /tab >}} +{{< tab name="Run Ollama outside of a container" >}} + +To run Ollama outside of a container: + +1. [Install](https://github.com/jmorganca/ollama) and run Ollama on your host + machine. +2. Pull the model to Ollama using the following command. + + ```console + $ ollama pull llama2 + ``` + +3. Remove the `ollama` service from the `docker-compose.yaml` and update properly the connection variables in `winy` service: + + ```diff + - OLLAMA=http://ollama:11434 + + OLLAMA= + ``` + +{{< /tab >}} +{{< /tabs >}} + +### Run your RAG application + +At this point, you have the following services in your Compose file: + +- Server service for your main RAG application +- Database service to store vectors in a Qdrant database +- (optional) Ollama service to run the LLM + service + +Once the application is running, open a browser and access the application at [http://localhost:8501](http://localhost:8501). + +Depending on your system and the LLM service that you chose, it may take several +minutes to answer. + +### Summary + +In this section, you learned how to set up a development environment to provide +access all the services that your GenAI application needs. + +Related information: + +- [Dockerfile reference](/reference/dockerfile.md) +- [Compose file reference](/reference/compose-file/_index.md) +- [Ollama Docker image](https://hub.docker.com/r/ollama/ollama) +- [GenAI Stack demo applications](https://github.com/docker/genai-stack) + +### Next steps + +See samples of more GenAI applications in the [GenAI Stack demo applications](https://github.com/docker/genai-stack). diff --git a/content/guides/rag-ollama/containerize.md b/content/guides/rag-ollama/containerize.md deleted file mode 100644 index 402c480af541..000000000000 --- a/content/guides/rag-ollama/containerize.md +++ /dev/null @@ -1,108 +0,0 @@ ---- -title: Containerize a RAG application -linkTitle: Containerize your app -weight: 10 -keywords: python, generative ai, genai, llm, ollama, containerize, initialize, qdrant -description: Learn how to containerize a RAG application. -aliases: - - /guides/use-case/rag-ollama/containerize/ ---- - -## Overview - -This section walks you through containerizing a RAG application using Docker. - -> [!NOTE] -> You can see more samples of containerized GenAI applications in the [GenAI Stack](https://github.com/docker/genai-stack) demo applications. - -## Get the sample application - -The sample application used in this guide is an example of RAG application, made by three main components, which are the building blocks for every RAG application. A Large Language Model hosted somewhere, in this case it is hosted in a container and served via [Ollama](https://ollama.ai/). A vector database, [Qdrant](https://qdrant.tech/), to store the embeddings of local data, and a web application, using [Streamlit](https://streamlit.io/) to offer the best user experience to the user. - -Clone the sample application. Open a terminal, change directory to a directory that you want to work in, and run the following command to clone the repository: - -```console -$ git clone https://github.com/mfranzon/winy.git -``` - -You should now have the following files in your `winy` directory. - -```text -├── winy/ -│ ├── .gitignore -│ ├── app/ -│ │ ├── main.py -│ │ ├── Dockerfile -| | └── requirements.txt -│ ├── tools/ -│ │ ├── create_db.py -│ │ ├── create_embeddings.py -│ │ ├── requirements.txt -│ │ ├── test.py -| | └── download_model.sh -│ ├── docker-compose.yaml -│ ├── wine_database.db -│ ├── LICENSE -│ └── README.md -``` - -## Containerizing your application: Essentials - -Containerizing an application involves packaging it along with its dependencies into a container, which ensures consistency across different environments. Here’s what you need to containerize an app like Winy : - -1. Dockerfile: A Dockerfile that contains instructions on how to build a Docker image for your application. It specifies the base image, dependencies, configuration files, and the command to run your application. - -2. Docker Compose File: Docker Compose is a tool for defining and running multi-container Docker applications. A Compose file allows you to configure your application's services, networks, and volumes in a single file. - -## Run the application - -Inside the `winy` directory, run the following command in a -terminal. - -```console -$ docker compose up --build -``` - -Docker builds and runs your application. Depending on your network connection, it may take several minutes to download all the dependencies. You'll see a message like the following in the terminal when the application is running. - -```console -server-1 | You can now view your Streamlit app in your browser. -server-1 | -server-1 | URL: http://0.0.0.0:8501 -server-1 | -``` - -Open a browser and view the application at [http://localhost:8501](http://localhost:8501). You should see a simple Streamlit application. - -The application requires a Qdrant database service and an LLM service to work properly. If you have access to services that you ran outside of Docker, specify the connection information in the `docker-compose.yaml`. - -```yaml -winy: - build: - context: ./app - dockerfile: Dockerfile - environment: - - QDRANT_CLIENT=http://qdrant:6333 # Specifies the url for the qdrant database - - OLLAMA=http://ollama:11434 # Specifies the url for the ollama service - container_name: winy - ports: - - "8501:8501" - depends_on: - - qdrant - - ollama -``` - -If you don't have the services running, continue with this guide to learn how you can run some or all of these services with Docker. -Remember that the `ollama` service is empty; it doesn't have any model. For this reason you need to pull a model before starting to use the RAG application. All the instructions are in the following page. - -In the terminal, press `ctrl`+`c` to stop the application. - -## Summary - -In this section, you learned how you can containerize and run your RAG -application using Docker. - -## Next steps - -In the next section, you'll learn how to properly configure the application with your preferred LLM model, completely locally, using Docker. - diff --git a/content/guides/rag-ollama/develop.md b/content/guides/rag-ollama/develop.md deleted file mode 100644 index 280da3295777..000000000000 --- a/content/guides/rag-ollama/develop.md +++ /dev/null @@ -1,165 +0,0 @@ ---- -title: Use containers for RAG development -linkTitle: Develop your app -weight: 10 -keywords: python, local, development, generative ai, genai, llm, rag, ollama -description: Learn how to develop your generative RAG application locally. -aliases: - - /guides/use-case/rag-ollama/develop/ ---- - -## Prerequisites - -Complete [Containerize a RAG application](containerize.md). - -## Overview - -In this section, you'll learn how to set up a development environment to access all the services that your generative RAG application needs. This includes: - -- Adding a local database -- Adding a local or remote LLM service - -> [!NOTE] -> You can see more samples of containerized GenAI applications in the [GenAI Stack](https://github.com/docker/genai-stack) demo applications. - -## Add a local database - -You can use containers to set up local services, like a database. In this section, you'll explore the database service in the `docker-compose.yaml` file. - -To run the database service: - -1. In the cloned repository's directory, open the `docker-compose.yaml` file in an IDE or text editor. - -2. In the `docker-compose.yaml` file, you'll see the following: - - ```yaml - services: - qdrant: - image: qdrant/qdrant - container_name: qdrant - ports: - - "6333:6333" - volumes: - - qdrant_data:/qdrant/storage - ``` - - > [!NOTE] - > To learn more about Qdrant, see the [Qdrant Official Docker Image](https://hub.docker.com/r/qdrant/qdrant). - -3. Start the application. Inside the `winy` directory, run the following command in a terminal. - - ```console - $ docker compose up --build - ``` - -4. Access the application. Open a browser and view the application at [http://localhost:8501](http://localhost:8501). You should see a simple Streamlit application. - -5. Stop the application. In the terminal, press `ctrl`+`c` to stop the application. - -## Add a local or remote LLM service - -The sample application supports both [Ollama](https://ollama.ai/). This guide provides instructions for the following scenarios: - -- Run Ollama in a container -- Run Ollama outside of a container - -While all platforms can use any of the previous scenarios, the performance and -GPU support may vary. You can use the following guidelines to help you choose the appropriate option: - -- Run Ollama in a container if you're on Linux, and using a native installation of the Docker Engine, or Windows 10/11, and using Docker Desktop, you - have a CUDA-supported GPU, and your system has at least 8 GB of RAM. -- Run Ollama outside of a container if running Docker Desktop on a Linux Machine. - -Choose one of the following options for your LLM service. - -{{< tabs >}} -{{< tab name="Run Ollama in a container" >}} - -When running Ollama in a container, you should have a CUDA-supported GPU. While you can run Ollama in a container without a supported GPU, the performance may not be acceptable. Only Linux and Windows 11 support GPU access to containers. - -To run Ollama in a container and provide GPU access: - -1. Install the prerequisites. - - For Docker Engine on Linux, install the [NVIDIA Container Toolkilt](https://github.com/NVIDIA/nvidia-container-toolkit). - - For Docker Desktop on Windows 10/11, install the latest [NVIDIA driver](https://www.nvidia.com/Download/index.aspx) and make sure you are using the [WSL2 backend](/manuals/desktop/features/wsl/_index.md#turn-on-docker-desktop-wsl-2) -2. The `docker-compose.yaml` file already contains the necessary instructions. In your own apps, you'll need to add the Ollama service in your `docker-compose.yaml`. The following is - the updated `docker-compose.yaml`: - - ```yaml - ollama: - image: ollama/ollama - container_name: ollama - ports: - - "8000:8000" - deploy: - resources: - reservations: - devices: - - driver: nvidia - count: 1 - capabilities: [gpu] - ``` - - > [!NOTE] - > For more details about the Compose instructions, see [Turn on GPU access with Docker Compose](/manuals/compose/how-tos/gpu-support.md). - -3. Once the Ollama container is up and running it is possible to use the `download_model.sh` inside the `tools` folder with this command: - - ```console - . ./download_model.sh - ``` - -Pulling an Ollama model can take several minutes. - -{{< /tab >}} -{{< tab name="Run Ollama outside of a container" >}} - -To run Ollama outside of a container: - -1. [Install](https://github.com/jmorganca/ollama) and run Ollama on your host - machine. -2. Pull the model to Ollama using the following command. - - ```console - $ ollama pull llama2 - ``` - -3. Remove the `ollama` service from the `docker-compose.yaml` and update properly the connection variables in `winy` service: - - ```diff - - OLLAMA=http://ollama:11434 - + OLLAMA= - ``` - -{{< /tab >}} -{{< /tabs >}} - -## Run your RAG application - -At this point, you have the following services in your Compose file: - -- Server service for your main RAG application -- Database service to store vectors in a Qdrant database -- (optional) Ollama service to run the LLM - service - -Once the application is running, open a browser and access the application at [http://localhost:8501](http://localhost:8501). - -Depending on your system and the LLM service that you chose, it may take several -minutes to answer. - -## Summary - -In this section, you learned how to set up a development environment to provide -access all the services that your GenAI application needs. - -Related information: - -- [Dockerfile reference](/reference/dockerfile.md) -- [Compose file reference](/reference/compose-file/_index.md) -- [Ollama Docker image](https://hub.docker.com/r/ollama/ollama) -- [GenAI Stack demo applications](https://github.com/docker/genai-stack) - -## Next steps - -See samples of more GenAI applications in the [GenAI Stack demo applications](https://github.com/docker/genai-stack). diff --git a/content/guides/reactjs/_index.md b/content/guides/reactjs/_index.md index 10024e063f5e..18feceef58a1 100644 --- a/content/guides/reactjs/_index.md +++ b/content/guides/reactjs/_index.md @@ -5,14 +5,18 @@ description: Containerize and develop React.js apps using Docker keywords: getting started, React.js, react.js, docker, language, Dockerfile summary: | This guide explains how to containerize React.js applications using Docker. -toc_min: 1 -toc_max: 2 -languages: [js] +aliases: + - /guides/reactjs/configure-github-actions/ + - /guides/reactjs/containerize/ + - /guides/reactjs/deploy/ + - /guides/reactjs/develop/ + - /guides/reactjs/run-tests/ params: + tags: [cicd] time: 20 minutes - --- + The React.js language-specific guide shows you how to containerize a React.js application using Docker, following best practices for creating efficient, production-ready containers. [React.js](https://react.dev/) is a widely used library for building interactive user interfaces. However, managing dependencies, environments, and deployments efficiently can be complex. Docker simplifies this process by providing a consistent and containerized environment. @@ -48,3 +52,1379 @@ Before you begin, make sure you're familiar with the following: - Understanding of Docker concepts such as images, containers, and Dockerfiles. If you're new to Docker, start with the [Docker basics](/get-started/docker-concepts/the-basics/what-is-a-container.md) guide. Once you've completed the React.js getting started modules, you’ll be ready to containerize your own React.js application using the examples and instructions provided in this guide. + +## Containerize a React.js Application + +### Prerequisites + +Before you begin, make sure the following tools are installed and available on your system: + +- You have installed the latest version of [Docker Desktop](/get-started/get-docker.md). +- You have a [git client](https://git-scm.com/downloads). The examples in this section use a command-line based git client, but you can use any client. + +> **New to Docker?** +> Start with the [Docker basics](/get-started/docker-concepts/the-basics/what-is-a-container.md) guide to get familiar with key concepts like images, containers, and Dockerfiles. + +--- + +### Overview + +This guide walks you through the complete process of containerizing a React.js application with Docker. You’ll learn how to create a production-ready Docker image using best practices that improve performance, security, scalability, and deployment efficiency. + +By the end of this guide, you will: + +- Containerize a React.js application using Docker. +- Create and optimize a Dockerfile for production builds. +- Use multi-stage builds to minimize image size. +- Serve the application efficiently with a custom NGINX configuration. +- Follow best practices for building secure and maintainable Docker images. + +--- + +### Get the sample application + +Clone the sample application to use with this guide. Open a terminal, change +directory to a directory that you want to work in, and run the following command +to clone the git repository: + +```console +$ git clone https://github.com/kristiyan-velkov/docker-reactjs-sample +``` +--- + +### Build the Docker image + +React.js is a front-end library that compiles into static assets, so the Dockerfile is tailored to optimize how React applications are built and served in a production environment. + +> [!TIP] +> +> [Gordon](/ai/gordon/), Docker's AI assistant, can generate Docker assets for your project. Ask Gordon to create a Dockerfile, Compose file, and `.dockerignore` tailored to your application. + +#### Step 1: Create the Dockerfile + +Before creating a Dockerfile, you need to choose a base image. You can either use the [Node.js Official Image](https://hub.docker.com/_/node) or a Docker Hardened Image (DHI) from the [Hardened Image catalog](https://hub.docker.com/hardened-images/catalog). + +Choosing DHI offers the advantage of a production-ready image that is lightweight and secure. For more information, see [Docker Hardened Images](https://docs.docker.com/dhi/). + +> [!IMPORTANT] +> This guide uses a stable Node.js LTS image tag that is considered secure when the guide is written. Because new releases and security patches are published regularly, the tag shown here may no longer be the safest option when you follow the guide. Always review the latest available image tags and select a secure, up-to-date version before building or deploying your application. +> +> Official Node.js Docker Images: https://hub.docker.com/_/node + +{{< tabs >}} +{{< tab name="Using Docker Hardened Images" >}} +Docker Hardened Images (DHIs) are available for Node.js in the [Docker Hardened Images catalog](https://hub.docker.com/hardened-images/catalog/dhi/node). Docker Hardened Images are freely available to everyone with no subscription required. You can pull and use them like any other Docker image after signing in to the DHI registry. For more information, see the [DHI quickstart](/dhi/get-started/) guide. + +1. Sign in to the DHI registry: + ```console + $ docker login dhi.io + ``` + +2. Pull the Node.js DHI (check the catalog for available versions): + ```console + $ docker pull dhi.io/node:24-alpine3.22-dev + ``` + +3. Pull the Nginx DHI (check the catalog for available versions): + ```console + $ docker pull dhi.io/nginx:1.28.0-alpine3.21-dev + ``` + +In the following Dockerfile, the `FROM` instructions use `dhi.io/node:24-alpine3.22-dev` and `dhi.io/nginx:1.28.0-alpine3.21-dev` as the base images. + +```dockerfile +# ========================================= +# Stage 1: Build the React.js Application +# ========================================= + +# Use a lightweight Node.js image for building (customizable via ARG) +FROM dhi.io/node:24-alpine3.22-dev AS builder + +# Set the working directory inside the container +WORKDIR /app + +# Copy package-related files first to leverage Docker's caching mechanism +COPY package.json package-lock.json* ./ + +# Install project dependencies using npm ci (ensures a clean, reproducible install) +RUN --mount=type=cache,target=/root/.npm npm ci + +# Copy the rest of the application source code into the container +COPY . . + +# Build the React.js application (outputs to /app/dist) +RUN npm run build + +# ========================================= +# Stage 2: Prepare Nginx to Serve Static Files +# ========================================= + +FROM dhi.io/nginx:1.28.0-alpine3.21-dev AS runner + +# Copy custom Nginx config +COPY nginx.conf /etc/nginx/nginx.conf + +# Copy the static build output from the build stage to Nginx's default HTML serving directory +COPY --chown=nginx:nginx --from=builder /app/dist /usr/share/nginx/html + +# Use a non-root user for security best practices +USER nginx + +# Expose port 8080 to allow HTTP traffic +# Note: The default NGINX container now listens on port 8080 instead of 80 +EXPOSE 8080 + +# Start Nginx directly with custom config +ENTRYPOINT ["nginx", "-c", "/etc/nginx/nginx.conf"] +CMD ["-g", "daemon off;"] +``` + +{{< /tab >}} +{{< tab name="Using the Docker Official Image" >}} + +Create a file named `Dockerfile` with the following contents: + +```dockerfile +# ========================================= +# Stage 1: Build the React.js Application +# ========================================= +ARG NODE_VERSION=24.12.0-alpine +ARG NGINX_VERSION=alpine3.22 + +# Use a lightweight Node.js image for building (customizable via ARG) +FROM node:${NODE_VERSION} AS builder + +# Set the working directory inside the container +WORKDIR /app + +# Copy package-related files first to leverage Docker's caching mechanism +COPY package.json package-lock.json* ./ + +# Install project dependencies using npm ci (ensures a clean, reproducible install) +RUN --mount=type=cache,target=/root/.npm npm ci + +# Copy the rest of the application source code into the container +COPY . . + +# Build the React.js application (outputs to /app/dist) +RUN npm run build + +# ========================================= +# Stage 2: Prepare Nginx to Serve Static Files +# ========================================= + +FROM nginxinc/nginx-unprivileged:${NGINX_VERSION} AS runner + +# Copy custom Nginx config +COPY nginx.conf /etc/nginx/nginx.conf + +# Copy the static build output from the build stage to Nginx's default HTML serving directory +COPY --chown=nginx:nginx --from=builder /app/dist /usr/share/nginx/html + +# Use a built-in non-root user for security best practices +USER nginx + +# Expose port 8080 to allow HTTP traffic +# Note: The default NGINX container now listens on port 8080 instead of 80 +EXPOSE 8080 + +# Start Nginx directly with custom config +ENTRYPOINT ["nginx", "-c", "/etc/nginx/nginx.conf"] +CMD ["-g", "daemon off;"] +``` + +> [!NOTE] +> We are using nginx-unprivileged instead of the standard NGINX image to follow security best practices. +> Running as a non-root user in the final image: +>- Reduces the attack surface +>- Aligns with Docker’s recommendations for container hardening +>- Helps comply with stricter security policies in production environments + +{{< /tab >}} +{{< /tabs >}} + +#### Step 2: Create the compose.yaml file + +Create a file named `compose.yaml` with the following contents: + +```yaml {collapse=true,title=compose.yaml} +services: + server: + build: + context: . + ports: + - 8080:8080 +``` + +#### Step 3: Create the .dockerignore file + +The `.dockerignore` file tells Docker which files and folders to exclude when building the image. + + +> [!NOTE] +>This helps: +>- Reduce image size +>- Speed up the build process +>- Prevent sensitive or unnecessary files (like `.env`, `.git`, or `node_modules`) from being added to the final image. +> +> To learn more, visit the [.dockerignore reference](/reference/dockerfile.md#dockerignore-file). + +Create a file named `.dockerignore` with the following contents: + +```dockerignore +# Ignore dependencies and build output +node_modules/ +dist/ +out/ +.tmp/ +.cache/ + +# Ignore Vite, Webpack, and React-specific build artifacts +.vite/ +.vitepress/ +.eslintcache +.npm/ +coverage/ +jest/ +cypress/ +cypress/screenshots/ +cypress/videos/ +reports/ + +# Ignore environment and config files (sensitive data) +*.env* +*.log + +# Ignore TypeScript build artifacts (if using TypeScript) +*.tsbuildinfo + +# Ignore lockfiles (optional if using Docker for package installation) +npm-debug.log* +yarn-debug.log* +yarn-error.log* +pnpm-debug.log* + +# Ignore local development files +.git/ +.gitignore +.vscode/ +.idea/ +*.swp +.DS_Store +Thumbs.db + +# Ignore Docker-related files (to avoid copying unnecessary configs) +Dockerfile +.dockerignore +docker-compose.yml +docker-compose.override.yml + +# Ignore build-specific cache files +*.lock + +``` + +#### Step 4: Create the `nginx.conf` file + +To serve your React.js application efficiently inside the container, you’ll configure NGINX with a custom setup. This configuration is optimized for performance, browser caching, gzip compression, and support for client-side routing. + +Create a file named `nginx.conf` in the root of your project directory, and add the following content: + +> [!NOTE] +> To learn more about configuring NGINX, see the [official NGINX documentation](https://nginx.org/en/docs/). + + +```nginx +worker_processes auto; + +# Store PID in /tmp (always writable) +pid /tmp/nginx.pid; + +events { + worker_connections 1024; +} + +http { + include /etc/nginx/mime.types; + default_type application/octet-stream; + + # Disable logging to avoid permission issues + access_log off; + error_log /dev/stderr warn; + + # Optimize static file serving + sendfile on; + tcp_nopush on; + tcp_nodelay on; + keepalive_timeout 65; + keepalive_requests 1000; + + # Gzip compression for optimized delivery + gzip on; + gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript image/svg+xml; + gzip_min_length 256; + gzip_vary on; + + server { + listen 8080; + server_name localhost; + + # Root directory where React.js build files are placed + root /usr/share/nginx/html; + index index.html; + + # Serve React.js static files with proper caching + location / { + try_files $uri /index.html; + } + + # Serve static assets with long cache expiration + location ~* \.(?:ico|css|js|gif|jpe?g|png|woff2?|eot|ttf|svg|map)$ { + expires 1y; + access_log off; + add_header Cache-Control "public, immutable"; + } + + # Handle React.js client-side routing + location /static/ { + expires 1y; + add_header Cache-Control "public, immutable"; + } + } +} +``` + +#### Step 5: Build the React.js application image + +With your custom configuration in place, you're now ready to build the Docker image for your React.js application. + +The updated setup includes: + +- Optimized browser caching and gzip compression +- Secure, non-root logging to avoid permission issues +- Support for React client-side routing by redirecting unmatched routes to `index.html` + +After completing the previous steps, your project directory should now contain the following files: + +```text +├── docker-reactjs-sample/ +│ ├── Dockerfile +│ ├── .dockerignore +│ ├── compose.yaml +│ └── nginx.conf +``` + +Now that your Dockerfile is configured, you can build the Docker image for your React.js application. + +> [!NOTE] +> The `docker build` command packages your application into an image using the instructions in the Dockerfile. It includes all necessary files from the current directory (called the [build context](/build/concepts/context/#what-is-a-build-context)). + +Run the following command from the root of your project: + +```console +$ docker build --tag docker-reactjs-sample . +``` + +What this command does: +- Uses the Dockerfile in the current directory (.) +- Packages the application and its dependencies into a Docker image +- Tags the image as docker-reactjs-sample so you can reference it later + + +#### Step 6: View local images + +After building your Docker image, you can check which images are available on your local machine using either the Docker CLI or [Docker Desktop](/manuals/desktop/use-desktop/images.md). Since you're already working in the terminal, let's use the Docker CLI. + +To list all locally available Docker images, run the following command: + +```console +$ docker images +``` + +Example Output: + +```shell +REPOSITORY TAG IMAGE ID CREATED SIZE +docker-reactjs-sample latest f39b47a97156 14 seconds ago 75.8MB +``` + +This output provides key details about your images: + +- **Repository** – The name assigned to the image. +- **Tag** – A version label that helps identify different builds (e.g., latest). +- **Image ID** – A unique identifier for the image. +- **Created** – The timestamp indicating when the image was built. +- **Size** – The total disk space used by the image. + +If the build was successful, you should see `docker-reactjs-sample` image listed. + +--- + +### Run the containerized application + +In the previous step, you created a Dockerfile for your React.js application and built a Docker image using the docker build command. Now it’s time to run that image in a container and verify that your application works as expected. + + +Inside the `docker-reactjs-sample` directory, run the following command in a +terminal. + +```console +$ docker compose up --build +``` + +Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see a simple React.js web application. + +Press `ctrl+c` in the terminal to stop your application. + +#### Run the application in the background + +You can run the application detached from the terminal by adding the `-d` +option. Inside the `docker-reactjs-sample` directory, run the following command +in a terminal. + +```console +$ docker compose up --build -d +``` + +Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see a simple web application preview. + + +To confirm that the container is running, use `docker ps` command: + +```console +$ docker ps +``` + +This will list all active containers along with their ports, names, and status. Look for a container exposing port 8080. + +Example Output: + +```shell +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +88bced6ade95 docker-reactjs-sample-server "nginx -c /etc/nginx…" About a minute ago Up About a minute 0.0.0.0:8080->8080/tcp docker-reactjs-sample-server-1 +``` + + +To stop the application, run: + +```console +$ docker compose down +``` + + +> [!NOTE] +> For more information about Compose commands, see the [Compose CLI +> reference](/reference/cli/docker/compose/). + +--- + +### Summary + +In this guide, you learned how to containerize, build, and run a React.js application using Docker. By following best practices, you created a secure, optimized, and production-ready setup. + +What you accomplished: +- Created a multi-stage `Dockerfile` that compiles the React.js application and serves the static files using Nginx. +- Created a `.dockerignore` file to exclude unnecessary files and keep the image clean and efficient. +- Built your Docker image using `docker build`. +- Ran the container using `docker compose up`, both in the foreground and in detached mode. +- Verified that the app was running by visiting [http://localhost:8080](http://localhost:8080). +- Learned how to stop the containerized application using `docker compose down`. + +You now have a fully containerized React.js application, running in a Docker container, and ready for deployment across any environment with confidence and consistency. + +--- + +### Related resources + +Explore official references and best practices to sharpen your Docker workflow: + +- [Multi-stage builds](/build/building/multi-stage/) – Learn how to separate build and runtime stages. +- [Best practices for writing Dockerfiles](/develop/develop-images/dockerfile_best-practices/) – Write efficient, maintainable, and secure Dockerfiles. +- [Build context in Docker](/build/concepts/context/) – Learn how context affects image builds. +- [`docker build` CLI reference](/reference/cli/docker/image/build/) – Build Docker images from a Dockerfile. +- [`docker images` CLI reference](/reference/cli/docker/image/ls/) – Manage and inspect local Docker images. +- [`docker compose up` CLI reference](/reference/cli/docker/compose/up/) – Start and run multi-container applications. +- [`docker compose down` CLI reference](/reference/cli/docker/compose/down/) – Stop and remove containers, networks, and volumes. + +--- + +### Next steps + +With your React.js application now containerized, you're ready to move on to the next step. + +In the next section, you'll learn how to develop your application using Docker containers, enabling a consistent, isolated, and reproducible development environment across any machine. + +## Use containers for React.js development + +### Prerequisites + +Complete [Containerize React.js application](containerize.md). + +--- + +### Overview + +In this section, you'll learn how to set up both production and development environments for your containerized React.js application using Docker Compose. This setup allows you to serve a static production build via Nginx and to develop efficiently inside containers using a live-reloading dev server with Compose Watch. + +You’ll learn how to: +- Configure separate containers for production and development +- Enable automatic file syncing using Compose Watch in development +- Debug and live-preview your changes in real-time without manual rebuilds + +--- + +### Automatically update services (development mode) + +Use Compose Watch to automatically sync source file changes into your containerized development environment. This provides a seamless, efficient development experience without needing to restart or rebuild containers manually. + +### Step 1: Create a development Dockerfile + +Create a file named `Dockerfile.dev` in your project root with the following content: + +```dockerfile +# ========================================= +# Stage 1: Develop the React.js Application +# ========================================= +ARG NODE_VERSION=24.12.0-alpine + +# Use a lightweight Node.js image for development +FROM node:${NODE_VERSION} AS dev + +# Set the working directory inside the container +WORKDIR /app + +# Copy package-related files first to leverage Docker's caching mechanism +COPY package.json package-lock.json* ./ + +# Install project dependencies +RUN --mount=type=cache,target=/root/.npm npm install + +# Copy the rest of the application source code into the container +COPY . . + +# Expose the port used by the Vite development server +EXPOSE 5173 + +# Use a default command, can be overridden in Docker compose.yml file +CMD ["npm", "run", "dev"] +``` + +This file sets up a lightweight development environment for your React app using the dev server. + + +#### Step 2: Update your `compose.yaml` file + +Open your `compose.yaml` file and define two services: one for production (`react-prod`) and one for development (`react-dev`). + +Here’s an example configuration for a React.js application: + +```yaml +services: + react-prod: + build: + context: . + dockerfile: Dockerfile + image: docker-reactjs-sample + ports: + - "8080:8080" + + react-dev: + build: + context: . + dockerfile: Dockerfile.dev + ports: + - "5173:5173" + develop: + watch: + - action: sync + path: . + target: /app + +``` +- The `react-prod` service builds and serves your static production app using Nginx. +- The `react-dev` service runs your React development server with live reload and hot module replacement. +- `watch` triggers file sync with Compose Watch. + +> [!NOTE] +> For more details, see the official guide: [Use Compose Watch](/manuals/compose/how-tos/file-watch.md). + +#### Step 3: Update vite.config.ts to ensure it works properly inside Docker + +To make Vite’s development server work reliably inside Docker, you need to update your vite.config.ts with the correct settings. + +Open the `vite.config.ts` file in your project root and update it as follows: + +```ts +/// + +import { defineConfig } from "vite"; +import react from "@vitejs/plugin-react"; + +export default defineConfig({ + base: "/", + plugins: [react()], + server: { + host: true, + port: 5173, + strictPort: true, + }, +}); +``` + +> [!NOTE] +> The `server` options in `vite.config.ts` are essential for running Vite inside Docker: +> - `host: true` allows the dev server to be accessible from outside the container. +> - `port: 5173` sets a consistent development port (must match the one exposed in Docker). +> - `strictPort: true` ensures Vite fails clearly if the port is unavailable, rather than switching silently. +> +> For full details, refer to the [Vite server configuration docs](https://vitejs.dev/config/server-options.html). + + +After completing the previous steps, your project directory should now contain the following files: + +```text +├── docker-reactjs-sample/ +│ ├── Dockerfile +│ ├── Dockerfile.dev +│ ├── .dockerignore +│ ├── compose.yaml +│ └── nginx.conf +``` + +#### Step 4: Start Compose Watch + +Run the following command from your project root to start your container in watch mode: + +```console +$ docker compose watch react-dev +``` + +#### Step 5: Test Compose Watch with React + +To verify that Compose Watch is working correctly: + +1. Open the `src/App.tsx` file in your text editor. + +2. Locate the following line: + + ```html +

    Vite + React

    + ``` + +3. Change it to: + + ```html +

    Hello from Docker Compose Watch

    + ``` + +4. Save the file. + +5. Open your browser at [http://localhost:5173](http://localhost:5173). + +You should see the updated text appear instantly, without needing to rebuild the container manually. This confirms that file watching and automatic synchronization are working as expected. + +--- + +### Summary + +In this section, you set up a complete development and production workflow for your React.js application using Docker and Docker Compose. + +Here's what you achieved: +- Created a `Dockerfile.dev` to streamline local development with hot reloading +- Defined separate `react-dev` and `react-prod` services in your `compose.yaml` file +- Enabled real-time file syncing using Compose Watch for a smoother development experience +- Verified that live updates work seamlessly by modifying and previewing a component + +With this setup, you're now equipped to build, run, and iterate on your React.js app entirely within containers—efficiently and consistently across environments. + +--- + +### Related resources + +Deepen your knowledge and improve your containerized development workflow with these guides: + +- [Using Compose Watch](/manuals/compose/how-tos/file-watch.md) – Automatically sync source changes during development +- [Multi-stage builds](/manuals/build/building/multi-stage.md) – Create efficient, production-ready Docker images +- [Dockerfile best practices](/build/building/best-practices/) – Write clean, secure, and optimized Dockerfiles. +- [Compose file reference](/compose/compose-file/) – Learn the full syntax and options available for configuring services in `compose.yaml`. +- [Docker volumes](/storage/volumes/) – Persist and manage data between container runs + +### Next steps + +In the next section, you'll learn how to run unit tests for your React.js application inside Docker containers. This ensures consistent testing across all environments and removes dependencies on local machine setup. + +## Run React.js tests in a container + +### Prerequisites + +Complete all the previous sections of this guide, starting with [Containerize React.js application](containerize.md). + +### Overview + +Testing is a critical part of the development process. In this section, you'll learn how to: + +- Run unit tests using Vitest inside a Docker container. +- Use Docker Compose to run tests in an isolated, reproducible environment. + +You’ll use [Vitest](https://vitest.dev) — a blazing fast test runner designed for Vite — along with [Testing Library](https://testing-library.com/) for assertions. + +--- + +### Run tests during development + +`docker-reactjs-sample` application includes a sample test file at location: + +```console +$ src/App.test.tsx +``` + +This file uses Vitest and React Testing Library to verify the behavior of `App` component. + +#### Step 1: Install Vitest and React Testing Library + +If you haven’t already added the necessary testing tools, install them by running: + +```console +$ npm install --save-dev vitest @testing-library/react @testing-library/jest-dom jsdom +``` + +Then, update the scripts section of your `package.json` file to include the following: + +```json +"scripts": { + "test": "vitest run" +} +``` + +--- + +#### Step 2: Configure Vitest + +Update `vitest.config.ts` file in your project root with the following configuration: + +```ts {hl_lines="14-18",linenos=true} +/// + +import { defineConfig } from "vite"; +import react from "@vitejs/plugin-react"; + +export default defineConfig({ + base: "/", + plugins: [react()], + server: { + host: true, + port: 5173, + strictPort: true, + }, + test: { + environment: "jsdom", + setupFiles: "./src/setupTests.ts", + globals: true, + }, +}); +``` + +> [!NOTE] +> The `test` options in `vitest.config.ts` are essential for reliable testing inside Docker: +> - `environment: "jsdom"` simulates a browser-like environment for rendering and DOM interactions. +> - `setupFiles: "./src/setupTests.ts"` loads global configuration or mocks before each test file (optional but recommended). +> - `globals: true` enables global test functions like `describe`, `it`, and `expect` without importing them. +> +> For more details, see the official [Vitest configuration docs](https://vitest.dev/config/). + +#### Step 3: Update compose.yaml + +Add a new service named `react-test` to your `compose.yaml` file. This service allows you to run your test suite in an isolated containerized environment. + +```yaml {hl_lines="22-26",linenos=true} +services: + react-dev: + build: + context: . + dockerfile: Dockerfile.dev + ports: + - "5173:5173" + develop: + watch: + - action: sync + path: . + target: /app + + react-prod: + build: + context: . + dockerfile: Dockerfile + image: docker-reactjs-sample + ports: + - "8080:8080" + + react-test: + build: + context: . + dockerfile: Dockerfile.dev + command: ["npm", "run", "test"] + +``` + +The react-test service reuses the same `Dockerfile.dev` used for [development](develop.md) and overrides the default command to run tests with `npm run test`. This setup ensures a consistent test environment that matches your local development configuration. + + +After completing the previous steps, your project directory should contain the following files: + +```text +├── docker-reactjs-sample/ +│ ├── Dockerfile +│ ├── Dockerfile.dev +│ ├── .dockerignore +│ ├── compose.yaml +│ └── nginx.conf +``` + +#### Step 4: Run the tests + +To execute your test suite inside the container, run the following command from your project root: + +```console +$ docker compose run --rm react-test +``` + +This command will: +- Start the `react-test` service defined in your `compose.yaml` file. +- Execute the `npm run test` script using the same environment as development. +- Automatically remove the container after the tests complete [`docker compose run --rm`](/reference/cli/docker/compose/run/) command. + +> [!NOTE] +> For more information about Compose commands, see the [Compose CLI +> reference](/reference/cli/docker/compose/). + +--- + +### Summary + +In this section, you learned how to run unit tests for your React.js application inside a Docker container using Vitest and Docker Compose. + +What you accomplished: +- Installed and configured Vitest and React Testing Library for testing React components. +- Created a `react-test` service in `compose.yaml` to isolate test execution. +- Reused the development `Dockerfile.dev` to ensure consistency between dev and test environments. +- Ran tests inside the container using `docker compose run --rm react-test`. +- Ensured reliable, repeatable testing across environments without relying on local machine setup. + +--- + +### Related resources + +Explore official references and best practices to sharpen your Docker testing workflow: + +- [Dockerfile reference](/reference/dockerfile/) – Understand all Dockerfile instructions and syntax. +- [Best practices for writing Dockerfiles](/develop/develop-images/dockerfile_best-practices/) – Write efficient, maintainable, and secure Dockerfiles. +- [Compose file reference](/compose/compose-file/) – Learn the full syntax and options available for configuring services in `compose.yaml`. +- [`docker compose run` CLI reference](/reference/cli/docker/compose/run/) – Run one-off commands in a service container. +--- + +### Next steps + +Next, you’ll learn how to set up a CI/CD pipeline using GitHub Actions to automatically build and test your React.js application in a containerized environment. This ensures your code is validated on every push or pull request, maintaining consistency and reliability across your development workflow. + +## Test your React.js deployment + +### Prerequisites + +Before you begin, make sure you’ve completed the following: +- Complete all the previous sections of this guide, starting with [Containerize React.js application](containerize.md). +- [Enable Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. + +> **New to Kubernetes?** +> Visit the [Kubernetes basics tutorial](https://kubernetes.io/docs/tutorials/kubernetes-basics/) to get familiar with how clusters, pods, deployments, and services work. + +--- + +### Overview + +This section guides you through deploying your containerized React.js application locally using [Docker Desktop’s built-in Kubernetes](/desktop/kubernetes/). Running your app in a local Kubernetes cluster allows you to closely simulate a real production environment, enabling you to test, validate, and debug your workloads with confidence before promoting them to staging or production. + +--- + +### Create a Kubernetes YAML file + +Follow these steps to define your deployment configuration: + +1. In the root of your project, create a new file named: reactjs-sample-kubernetes.yaml + +2. Open the file in your IDE or preferred text editor. + +3. Add the following configuration, and be sure to replace `{DOCKER_USERNAME}` and `{DOCKERHUB_PROJECT_NAME}` with your actual Docker Hub username and repository name from the previous [Automate your builds with GitHub Actions](configure-github-actions.md). + + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: reactjs-sample + namespace: default +spec: + replicas: 1 + selector: + matchLabels: + app: reactjs-sample + template: + metadata: + labels: + app: reactjs-sample + spec: + containers: + - name: reactjs-container + image: {DOCKER_USERNAME}/{DOCKERHUB_PROJECT_NAME}:latest + imagePullPolicy: Always + ports: + - containerPort: 8080 +--- +apiVersion: v1 +kind: Service +metadata: + name: reactjs-sample-service + namespace: default +spec: + type: NodePort + selector: + app: reactjs-sample + ports: + - port: 8080 + targetPort: 8080 + nodePort: 30001 +``` + +This manifest defines two key Kubernetes resources, separated by `---`: + +- Deployment + Deploys a single replica of your React.js application inside a pod. The pod uses the Docker image built and pushed by your GitHub Actions CI/CD workflow + (refer to [Automate your builds with GitHub Actions](configure-github-actions.md)). + The container listens on port `8080`, which is typically used by [Nginx](https://nginx.org/en/docs/) to serve your production React app. + +- Service (NodePort) + Exposes the deployed pod to your local machine. + It forwards traffic from port `30001` on your host to port `8080` inside the container. + This lets you access the application in your browser at [http://localhost:30001](http://localhost:30001). + +> [!NOTE] +> To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). + +--- + +### Deploy and check your application + +Follow these steps to deploy your containerized React.js app into a local Kubernetes cluster and verify that it’s running correctly. + +#### Step 1. Apply the Kubernetes configuration + +In your terminal, navigate to the directory where your `reactjs-sample-kubernetes.yaml` file is located, then deploy the resources using: + +```console + $ kubectl apply -f reactjs-sample-kubernetes.yaml +``` + +If everything is configured properly, you’ll see confirmation that both the Deployment and the Service were created: + +```shell + deployment.apps/reactjs-sample created + service/reactjs-sample-service created +``` + +This output means that both the Deployment and the Service were successfully created and are now running inside your local cluster. + +#### Step 2. Check the deployment status + +Run the following command to check the status of your deployment: + +```console + $ kubectl get deployments +``` + +You should see an output similar to: + +```shell + NAME READY UP-TO-DATE AVAILABLE AGE + reactjs-sample 1/1 1 1 14s +``` + +This confirms that your pod is up and running with one replica available. + +#### Step 3. Verify the service exposure + +Check if the NodePort service is exposing your app to your local machine: + +```console +$ kubectl get services +``` + +You should see something like: + +```shell +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +reactjs-sample-service NodePort 10.100.244.65 8080:30001/TCP 1m +``` + +This output confirms that your app is available via NodePort on port 30001. + +#### Step 4. Access your app in the browser + +Open your browser and navigate to [http://localhost:30001](http://localhost:30001). + +You should see your production-ready React.js Sample application running — served by your local Kubernetes cluster. + +#### Step 5. Clean up Kubernetes resources + +Once you're done testing, you can delete the deployment and service using: + +```console + $ kubectl delete -f reactjs-sample-kubernetes.yaml +``` + +Expected output: + +```shell + deployment.apps "reactjs-sample" deleted + service "reactjs-sample-service" deleted +``` + +This ensures your cluster stays clean and ready for the next deployment. + +--- + +### Summary + +In this section, you learned how to deploy your React.js application to a local Kubernetes cluster using Docker Desktop. This setup allows you to test and debug your containerized app in a production-like environment before deploying it to the cloud. + +What you accomplished: + +- Created a Kubernetes Deployment and NodePort Service for your React.js app +- Used `kubectl apply` to deploy the application locally +- Verified the app was running and accessible at `http://localhost:30001` +- Cleaned up your Kubernetes resources after testing + +--- + +### Related resources + +Explore official references and best practices to sharpen your Kubernetes deployment workflow: + +- [Kubernetes documentation](https://kubernetes.io/docs/home/) – Learn about core concepts, workloads, services, and more. +- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) – Use Docker Desktop’s built-in Kubernetes support for local testing and development. +- [`kubectl` CLI reference](https://kubernetes.io/docs/reference/kubectl/) – Manage Kubernetes clusters from the command line. +- [Kubernetes Deployment resource](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) – Understand how to manage and scale applications using Deployments. +- [Kubernetes Service resource](https://kubernetes.io/docs/concepts/services-networking/service/) – Learn how to expose your application to internal and external traffic. + +## Automate your builds with GitHub Actions + +### Prerequisites + +Complete all the previous sections of this guide, starting with [Containerize React.js application](containerize.md). + +You must also have: +- A [GitHub](https://github.com/signup) account. +- A verified [Docker Hub](https://hub.docker.com/signup) account. + +--- + +### Overview + +In this section, you'll set up a CI/CD pipeline using [GitHub Actions](https://docs.github.com/en/actions) to automatically: + +- Build your React.js application inside a Docker container. +- Run tests in a consistent environment. +- Push the production-ready image to [Docker Hub](https://hub.docker.com). + +--- + +### Connect your GitHub repository to Docker Hub + +To enable GitHub Actions to build and push Docker images, you’ll securely store your Docker Hub credentials in your new GitHub repository. + +#### Step 1: Connect your GitHub repository to Docker Hub + +1. Create a Personal Access Token (PAT) from [Docker Hub](https://hub.docker.com) + 1. Go to your **Docker Hub account → Account Settings → Security**. + 2. Generate a new Access Token with **Read/Write** permissions. + 3. Name it something like `docker-reactjs-sample`. + 4. Copy and save the token — you’ll need it in Step 4. + +2. Create a repository in [Docker Hub](https://hub.docker.com/repositories/) + 1. Go to your **Docker Hub account → Create a repository**. + 2. For the Repository Name, use something descriptive — for example: `reactjs-sample`. + 3. Once created, copy and save the repository name — you’ll need it in Step 4. + +3. Create a new [GitHub repository](https://github.com/new) for your React.js project + +4. Add Docker Hub credentials as GitHub repository secrets + + In your newly created GitHub repository: + + 1. Navigate to: + **Settings → Secrets and variables → Actions → New repository secret**. + + 2. Add the following secrets: + + | Name | Value | + |-------------------|--------------------------------| + | `DOCKER_USERNAME` | Your Docker Hub username | + | `DOCKERHUB_TOKEN` | Your Docker Hub access token (created in Step 1) | + | `DOCKERHUB_PROJECT_NAME` | Your Docker Project Name (created in Step 2) | + + These secrets let GitHub Actions to authenticate securely with Docker Hub during automated workflows. + +5. Connect Your Local Project to GitHub + + Link your local project `docker-reactjs-sample` to the GitHub repository you just created by running the following command from your project root: + + ```console + $ git remote set-url origin https://github.com/{your-username}/{your-repository-name}.git + ``` + + >[!IMPORTANT] + >Replace `{your-username}` and `{your-repository}` with your actual GitHub username and repository name. + + To confirm that your local project is correctly connected to the remote GitHub repository, run: + + ```console + $ git remote -v + ``` + + You should see output similar to: + + ```console + origin https://github.com/{your-username}/{your-repository-name}.git (fetch) + origin https://github.com/{your-username}/{your-repository-name}.git (push) + ``` + + This confirms that your local repository is properly linked and ready to push your source code to GitHub. + +6. Push Your Source Code to GitHub + + Follow these steps to commit and push your local project to your GitHub repository: + + 1. Stage all files for commit. + + ```console + $ git add -A + ``` + This command stages all changes — including new, modified, and deleted files — preparing them for commit. + + + 2. Commit your changes. + + ```console + $ git commit -m "Initial commit" + ``` + This command creates a commit that snapshots the staged changes with a descriptive message. + + 3. Push the code to the `main` branch. + + ```console + $ git push -u origin main + ``` + This command pushes your local commits to the `main` branch of the remote GitHub repository and sets the upstream branch. + +Once completed, your code will be available on GitHub, and any GitHub Actions workflow you’ve configured will run automatically. + +> [!NOTE] +> Learn more about the Git commands used in this step: +> - [Git add](https://git-scm.com/docs/git-add) – Stage changes (new, modified, deleted) for commit +> - [Git commit](https://git-scm.com/docs/git-commit) – Save a snapshot of your staged changes +> - [Git push](https://git-scm.com/docs/git-push) – Upload local commits to your GitHub repository +> - [Git remote](https://git-scm.com/docs/git-remote) – View and manage remote repository URLs + +--- + +#### Step 2: Set up the workflow + +Now you'll create a GitHub Actions workflow that builds your Docker image, runs tests, and pushes the image to Docker Hub. + +1. Go to your repository on GitHub and select the **Actions** tab in the top menu. + +2. Select **Set up a workflow yourself** when prompted. + + This opens an inline editor to create a new workflow file. By default, it will be saved to: + `.github/workflows/main.yml` + + +3. Add the following workflow configuration to the new file: + +```yaml +name: CI/CD – React.js Application with Docker + +on: + push: + branches: [main] + pull_request: + branches: [main] + types: [opened, synchronize, reopened] + +jobs: + build-test-push: + name: Build, Test and Push Docker Image + runs-on: ubuntu-latest + + steps: + # 1. Checkout source code + - name: Checkout source code + uses: actions/checkout@{{% param "checkout_action_version" %}} + with: + fetch-depth: 0 # Fetches full history for better caching/context + + # 2. Set up Docker Buildx + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} + + # 3. Cache Docker layers + - name: Cache Docker layers + uses: actions/cache@{{% param "cache_action_version" %}} + with: + path: /tmp/.buildx-cache + key: ${{ runner.os }}-buildx-${{ github.sha }} + restore-keys: ${{ runner.os }}-buildx- + + # 4. Cache npm dependencies + - name: Cache npm dependencies + uses: actions/cache@{{% param "cache_action_version" %}} + with: + path: ~/.npm + key: ${{ runner.os }}-npm-${{ hashFiles('**/package-lock.json') }} + restore-keys: ${{ runner.os }}-npm- + + # 5. Extract metadata + - name: Extract metadata + id: meta + run: | + echo "REPO_NAME=${GITHUB_REPOSITORY##*/}" >> "$GITHUB_OUTPUT" + echo "SHORT_SHA=${GITHUB_SHA::7}" >> "$GITHUB_OUTPUT" + + # 6. Build dev Docker image + - name: Build Docker image for tests + uses: docker/build-push-action@{{% param "build_push_action_version" %}} + with: + context: . + file: Dockerfile.dev + tags: ${{ steps.meta.outputs.REPO_NAME }}-dev:latest + load: true # Load to local Docker daemon for testing + cache-from: type=local,src=/tmp/.buildx-cache + cache-to: type=local,dest=/tmp/.buildx-cache,mode=max + + # 7. Run Vitest tests + - name: Run Vitest tests and generate report + run: | + docker run --rm \ + --workdir /app \ + --entrypoint "" \ + ${{ steps.meta.outputs.REPO_NAME }}-dev:latest \ + sh -c "npm ci && npx vitest run --reporter=verbose" + env: + CI: true + NODE_ENV: test + timeout-minutes: 10 + + # 8. Login to Docker Hub + - name: Log in to Docker Hub + uses: docker/login-action@{{% param "login_action_version" %}} + with: + username: ${{ secrets.DOCKER_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + + # 9. Build and push prod image + - name: Build and push production image + uses: docker/build-push-action@{{% param "build_push_action_version" %}} + with: + context: . + file: Dockerfile + push: true + platforms: linux/amd64,linux/arm64 + tags: | + ${{ secrets.DOCKER_USERNAME }}/${{ secrets.DOCKERHUB_PROJECT_NAME }}:latest + ${{ secrets.DOCKER_USERNAME }}/${{ secrets.DOCKERHUB_PROJECT_NAME }}:${{ steps.meta.outputs.SHORT_SHA }} + cache-from: type=local,src=/tmp/.buildx-cache +``` + +This workflow performs the following tasks for your React.js application: +- Triggers on every `push` or `pull request` targeting the `main` branch. +- Builds a development Docker image using `Dockerfile.dev`, optimized for testing. +- Executes unit tests using Vitest inside a clean, containerized environment to ensure consistency. +- Halts the workflow immediately if any test fails — enforcing code quality. +- Caches both Docker build layers and npm dependencies for faster CI runs. +- Authenticates securely with Docker Hub using GitHub repository secrets. +- Builds a production-ready image using the `prod` stage in `Dockerfile`. +- Tags and pushes the final image to Docker Hub with both `latest` and short SHA tags for traceability. + +> [!NOTE] +> For more information about `docker/build-push-action`, refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). + +--- + +#### Step 3: Run the workflow + +After you've added your workflow file, it's time to trigger and observe the CI/CD process in action. + +1. Commit and push your workflow file + + Select "Commit changes…" in the GitHub editor. + + - This push will automatically trigger the GitHub Actions pipeline. + +2. Monitor the workflow execution + + 1. Go to the Actions tab in your GitHub repository. + 2. Select the workflow run to follow each step: `build`, `test`, and (if successful) `push`. + +3. Verify the Docker image on Docker Hub + + - After a successful workflow run, visit your [Docker Hub repositories](https://hub.docker.com/repositories). + - You should see a new image under your repository with: + - Repository name: `${your-repository-name}` + - Tags include: + - `latest` – represents the most recent successful build; ideal for quick testing or deployment. + - `` – a unique identifier based on the commit hash, useful for version tracking, rollbacks, and traceability. + +> [!TIP] Protect your main branch +> To maintain code quality and prevent accidental direct pushes, enable branch protection rules: +> - Navigate to your **GitHub repo → Settings → Branches**. +> - Under Branch protection rules, click **Add rule**. +> - Specify `main` as the branch name. +> - Enable options like: +> - *Require a pull request before merging*. +> - *Require status checks to pass before merging*. +> +> This ensures that only tested and reviewed code is merged into `main` branch. +--- + +### Summary + +In this section, you set up a complete CI/CD pipeline for your containerized React.js application using GitHub Actions. + +Here's what you accomplished: + +- Created a new GitHub repository specifically for your project. +- Generated a secure Docker Hub access token and added it to GitHub as a secret. +- Defined a GitHub Actions workflow to: + - Build your application inside a Docker container. + - Run tests in a consistent, containerized environment. + - Push a production-ready image to Docker Hub if tests pass. +- Triggered and verified the workflow execution through GitHub Actions. +- Confirmed that your image was successfully published to Docker Hub. + +With this setup, your React.js application is now ready for automated testing and deployment across environments — increasing confidence, consistency, and team productivity. + +--- + +### Related resources + +Deepen your understanding of automation and best practices for containerized apps: + +- [Introduction to GitHub Actions](/guides/gha.md) – Learn how GitHub Actions automate your workflows +- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) – Set up container builds with GitHub Actions +- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) – Full reference for writing GitHub workflows +- [Compose file reference](/compose/compose-file/) – Full configuration reference for `compose.yaml` +- [Best practices for writing Dockerfiles](/develop/develop-images/dockerfile_best-practices/) – Optimize your image for performance and security + +--- + +### Next steps + +Next, learn how you can locally test and debug your React.js workloads on Kubernetes before deploying. This helps you ensure your application behaves as expected in a production-like environment, reducing surprises during deployment. diff --git a/content/guides/reactjs/configure-github-actions.md b/content/guides/reactjs/configure-github-actions.md deleted file mode 100644 index 158506c13462..000000000000 --- a/content/guides/reactjs/configure-github-actions.md +++ /dev/null @@ -1,321 +0,0 @@ ---- -title: Automate your builds with GitHub Actions -linkTitle: GitHub Actions CI -weight: 60 -keywords: CI/CD, GitHub( Actions), React.js, Next.js -description: Learn how to configure CI/CD using GitHub Actions for your React.js application. - ---- - -## Prerequisites - -Complete all the previous sections of this guide, starting with [Containerize React.js application](containerize.md). - -You must also have: -- A [GitHub](https://github.com/signup) account. -- A verified [Docker Hub](https://hub.docker.com/signup) account. - ---- - -## Overview - -In this section, you'll set up a CI/CD pipeline using [GitHub Actions](https://docs.github.com/en/actions) to automatically: - -- Build your React.js application inside a Docker container. -- Run tests in a consistent environment. -- Push the production-ready image to [Docker Hub](https://hub.docker.com). - ---- - -## Connect your GitHub repository to Docker Hub - -To enable GitHub Actions to build and push Docker images, you’ll securely store your Docker Hub credentials in your new GitHub repository. - -### Step 1: Connect your GitHub repository to Docker Hub - -1. Create a Personal Access Token (PAT) from [Docker Hub](https://hub.docker.com) - 1. Go to your **Docker Hub account → Account Settings → Security**. - 2. Generate a new Access Token with **Read/Write** permissions. - 3. Name it something like `docker-reactjs-sample`. - 4. Copy and save the token — you’ll need it in Step 4. - -2. Create a repository in [Docker Hub](https://hub.docker.com/repositories/) - 1. Go to your **Docker Hub account → Create a repository**. - 2. For the Repository Name, use something descriptive — for example: `reactjs-sample`. - 3. Once created, copy and save the repository name — you’ll need it in Step 4. - -3. Create a new [GitHub repository](https://github.com/new) for your React.js project - -4. Add Docker Hub credentials as GitHub repository secrets - - In your newly created GitHub repository: - - 1. Navigate to: - **Settings → Secrets and variables → Actions → New repository secret**. - - 2. Add the following secrets: - - | Name | Value | - |-------------------|--------------------------------| - | `DOCKER_USERNAME` | Your Docker Hub username | - | `DOCKERHUB_TOKEN` | Your Docker Hub access token (created in Step 1) | - | `DOCKERHUB_PROJECT_NAME` | Your Docker Project Name (created in Step 2) | - - These secrets let GitHub Actions to authenticate securely with Docker Hub during automated workflows. - -5. Connect Your Local Project to GitHub - - Link your local project `docker-reactjs-sample` to the GitHub repository you just created by running the following command from your project root: - - ```console - $ git remote set-url origin https://github.com/{your-username}/{your-repository-name}.git - ``` - - >[!IMPORTANT] - >Replace `{your-username}` and `{your-repository}` with your actual GitHub username and repository name. - - To confirm that your local project is correctly connected to the remote GitHub repository, run: - - ```console - $ git remote -v - ``` - - You should see output similar to: - - ```console - origin https://github.com/{your-username}/{your-repository-name}.git (fetch) - origin https://github.com/{your-username}/{your-repository-name}.git (push) - ``` - - This confirms that your local repository is properly linked and ready to push your source code to GitHub. - -6. Push Your Source Code to GitHub - - Follow these steps to commit and push your local project to your GitHub repository: - - 1. Stage all files for commit. - - ```console - $ git add -A - ``` - This command stages all changes — including new, modified, and deleted files — preparing them for commit. - - - 2. Commit your changes. - - ```console - $ git commit -m "Initial commit" - ``` - This command creates a commit that snapshots the staged changes with a descriptive message. - - 3. Push the code to the `main` branch. - - ```console - $ git push -u origin main - ``` - This command pushes your local commits to the `main` branch of the remote GitHub repository and sets the upstream branch. - -Once completed, your code will be available on GitHub, and any GitHub Actions workflow you’ve configured will run automatically. - -> [!NOTE] -> Learn more about the Git commands used in this step: -> - [Git add](https://git-scm.com/docs/git-add) – Stage changes (new, modified, deleted) for commit -> - [Git commit](https://git-scm.com/docs/git-commit) – Save a snapshot of your staged changes -> - [Git push](https://git-scm.com/docs/git-push) – Upload local commits to your GitHub repository -> - [Git remote](https://git-scm.com/docs/git-remote) – View and manage remote repository URLs - ---- - -### Step 2: Set up the workflow - -Now you'll create a GitHub Actions workflow that builds your Docker image, runs tests, and pushes the image to Docker Hub. - -1. Go to your repository on GitHub and select the **Actions** tab in the top menu. - -2. Select **Set up a workflow yourself** when prompted. - - This opens an inline editor to create a new workflow file. By default, it will be saved to: - `.github/workflows/main.yml` - - -3. Add the following workflow configuration to the new file: - -```yaml -name: CI/CD – React.js Application with Docker - -on: - push: - branches: [main] - pull_request: - branches: [main] - types: [opened, synchronize, reopened] - -jobs: - build-test-push: - name: Build, Test and Push Docker Image - runs-on: ubuntu-latest - - steps: - # 1. Checkout source code - - name: Checkout source code - uses: actions/checkout@{{% param "checkout_action_version" %}} - with: - fetch-depth: 0 # Fetches full history for better caching/context - - # 2. Set up Docker Buildx - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} - - # 3. Cache Docker layers - - name: Cache Docker layers - uses: actions/cache@{{% param "cache_action_version" %}} - with: - path: /tmp/.buildx-cache - key: ${{ runner.os }}-buildx-${{ github.sha }} - restore-keys: ${{ runner.os }}-buildx- - - # 4. Cache npm dependencies - - name: Cache npm dependencies - uses: actions/cache@{{% param "cache_action_version" %}} - with: - path: ~/.npm - key: ${{ runner.os }}-npm-${{ hashFiles('**/package-lock.json') }} - restore-keys: ${{ runner.os }}-npm- - - # 5. Extract metadata - - name: Extract metadata - id: meta - run: | - echo "REPO_NAME=${GITHUB_REPOSITORY##*/}" >> "$GITHUB_OUTPUT" - echo "SHORT_SHA=${GITHUB_SHA::7}" >> "$GITHUB_OUTPUT" - - # 6. Build dev Docker image - - name: Build Docker image for tests - uses: docker/build-push-action@{{% param "build_push_action_version" %}} - with: - context: . - file: Dockerfile.dev - tags: ${{ steps.meta.outputs.REPO_NAME }}-dev:latest - load: true # Load to local Docker daemon for testing - cache-from: type=local,src=/tmp/.buildx-cache - cache-to: type=local,dest=/tmp/.buildx-cache,mode=max - - # 7. Run Vitest tests - - name: Run Vitest tests and generate report - run: | - docker run --rm \ - --workdir /app \ - --entrypoint "" \ - ${{ steps.meta.outputs.REPO_NAME }}-dev:latest \ - sh -c "npm ci && npx vitest run --reporter=verbose" - env: - CI: true - NODE_ENV: test - timeout-minutes: 10 - - # 8. Login to Docker Hub - - name: Log in to Docker Hub - uses: docker/login-action@{{% param "login_action_version" %}} - with: - username: ${{ secrets.DOCKER_USERNAME }} - password: ${{ secrets.DOCKERHUB_TOKEN }} - - # 9. Build and push prod image - - name: Build and push production image - uses: docker/build-push-action@{{% param "build_push_action_version" %}} - with: - context: . - file: Dockerfile - push: true - platforms: linux/amd64,linux/arm64 - tags: | - ${{ secrets.DOCKER_USERNAME }}/${{ secrets.DOCKERHUB_PROJECT_NAME }}:latest - ${{ secrets.DOCKER_USERNAME }}/${{ secrets.DOCKERHUB_PROJECT_NAME }}:${{ steps.meta.outputs.SHORT_SHA }} - cache-from: type=local,src=/tmp/.buildx-cache -``` - -This workflow performs the following tasks for your React.js application: -- Triggers on every `push` or `pull request` targeting the `main` branch. -- Builds a development Docker image using `Dockerfile.dev`, optimized for testing. -- Executes unit tests using Vitest inside a clean, containerized environment to ensure consistency. -- Halts the workflow immediately if any test fails — enforcing code quality. -- Caches both Docker build layers and npm dependencies for faster CI runs. -- Authenticates securely with Docker Hub using GitHub repository secrets. -- Builds a production-ready image using the `prod` stage in `Dockerfile`. -- Tags and pushes the final image to Docker Hub with both `latest` and short SHA tags for traceability. - -> [!NOTE] -> For more information about `docker/build-push-action`, refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). - ---- - -### Step 3: Run the workflow - -After you've added your workflow file, it's time to trigger and observe the CI/CD process in action. - -1. Commit and push your workflow file - - Select "Commit changes…" in the GitHub editor. - - - This push will automatically trigger the GitHub Actions pipeline. - -2. Monitor the workflow execution - - 1. Go to the Actions tab in your GitHub repository. - 2. Select the workflow run to follow each step: `build`, `test`, and (if successful) `push`. - -3. Verify the Docker image on Docker Hub - - - After a successful workflow run, visit your [Docker Hub repositories](https://hub.docker.com/repositories). - - You should see a new image under your repository with: - - Repository name: `${your-repository-name}` - - Tags include: - - `latest` – represents the most recent successful build; ideal for quick testing or deployment. - - `` – a unique identifier based on the commit hash, useful for version tracking, rollbacks, and traceability. - -> [!TIP] Protect your main branch -> To maintain code quality and prevent accidental direct pushes, enable branch protection rules: -> - Navigate to your **GitHub repo → Settings → Branches**. -> - Under Branch protection rules, click **Add rule**. -> - Specify `main` as the branch name. -> - Enable options like: -> - *Require a pull request before merging*. -> - *Require status checks to pass before merging*. -> -> This ensures that only tested and reviewed code is merged into `main` branch. ---- - -## Summary - -In this section, you set up a complete CI/CD pipeline for your containerized React.js application using GitHub Actions. - -Here's what you accomplished: - -- Created a new GitHub repository specifically for your project. -- Generated a secure Docker Hub access token and added it to GitHub as a secret. -- Defined a GitHub Actions workflow to: - - Build your application inside a Docker container. - - Run tests in a consistent, containerized environment. - - Push a production-ready image to Docker Hub if tests pass. -- Triggered and verified the workflow execution through GitHub Actions. -- Confirmed that your image was successfully published to Docker Hub. - -With this setup, your React.js application is now ready for automated testing and deployment across environments — increasing confidence, consistency, and team productivity. - ---- - -## Related resources - -Deepen your understanding of automation and best practices for containerized apps: - -- [Introduction to GitHub Actions](/guides/gha.md) – Learn how GitHub Actions automate your workflows -- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) – Set up container builds with GitHub Actions -- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) – Full reference for writing GitHub workflows -- [Compose file reference](/compose/compose-file/) – Full configuration reference for `compose.yaml` -- [Best practices for writing Dockerfiles](/develop/develop-images/dockerfile_best-practices/) – Optimize your image for performance and security - ---- - -## Next steps - -Next, learn how you can locally test and debug your React.js workloads on Kubernetes before deploying. This helps you ensure your application behaves as expected in a production-like environment, reducing surprises during deployment. diff --git a/content/guides/reactjs/containerize.md b/content/guides/reactjs/containerize.md deleted file mode 100644 index dd5a5629e6ce..000000000000 --- a/content/guides/reactjs/containerize.md +++ /dev/null @@ -1,508 +0,0 @@ ---- -title: Containerize a React.js Application -linkTitle: Containerize -weight: 10 -keywords: react.js, node, image, initialize, build -description: Learn how to containerize a React.js application with Docker by creating an optimized, production-ready image using best practices for performance, security, and scalability. - ---- - -## Prerequisites - -Before you begin, make sure the following tools are installed and available on your system: - -- You have installed the latest version of [Docker Desktop](/get-started/get-docker.md). -- You have a [git client](https://git-scm.com/downloads). The examples in this section use a command-line based git client, but you can use any client. - -> **New to Docker?** -> Start with the [Docker basics](/get-started/docker-concepts/the-basics/what-is-a-container.md) guide to get familiar with key concepts like images, containers, and Dockerfiles. - ---- - -## Overview - -This guide walks you through the complete process of containerizing a React.js application with Docker. You’ll learn how to create a production-ready Docker image using best practices that improve performance, security, scalability, and deployment efficiency. - -By the end of this guide, you will: - -- Containerize a React.js application using Docker. -- Create and optimize a Dockerfile for production builds. -- Use multi-stage builds to minimize image size. -- Serve the application efficiently with a custom NGINX configuration. -- Follow best practices for building secure and maintainable Docker images. - ---- - -## Get the sample application - -Clone the sample application to use with this guide. Open a terminal, change -directory to a directory that you want to work in, and run the following command -to clone the git repository: - -```console -$ git clone https://github.com/kristiyan-velkov/docker-reactjs-sample -``` ---- - -## Build the Docker image - -React.js is a front-end library that compiles into static assets, so the Dockerfile is tailored to optimize how React applications are built and served in a production environment. - -> [!TIP] -> -> [Gordon](/ai/gordon/), Docker's AI assistant, can generate Docker assets for your project. Ask Gordon to create a Dockerfile, Compose file, and `.dockerignore` tailored to your application. - -### Step 1: Create the Dockerfile - -Before creating a Dockerfile, you need to choose a base image. You can either use the [Node.js Official Image](https://hub.docker.com/_/node) or a Docker Hardened Image (DHI) from the [Hardened Image catalog](https://hub.docker.com/hardened-images/catalog). - -Choosing DHI offers the advantage of a production-ready image that is lightweight and secure. For more information, see [Docker Hardened Images](https://docs.docker.com/dhi/). - -> [!IMPORTANT] -> This guide uses a stable Node.js LTS image tag that is considered secure when the guide is written. Because new releases and security patches are published regularly, the tag shown here may no longer be the safest option when you follow the guide. Always review the latest available image tags and select a secure, up-to-date version before building or deploying your application. -> -> Official Node.js Docker Images: https://hub.docker.com/_/node - -{{< tabs >}} -{{< tab name="Using Docker Hardened Images" >}} -Docker Hardened Images (DHIs) are available for Node.js in the [Docker Hardened Images catalog](https://hub.docker.com/hardened-images/catalog/dhi/node). Docker Hardened Images are freely available to everyone with no subscription required. You can pull and use them like any other Docker image after signing in to the DHI registry. For more information, see the [DHI quickstart](/dhi/get-started/) guide. - -1. Sign in to the DHI registry: - ```console - $ docker login dhi.io - ``` - -2. Pull the Node.js DHI (check the catalog for available versions): - ```console - $ docker pull dhi.io/node:24-alpine3.22-dev - ``` - -3. Pull the Nginx DHI (check the catalog for available versions): - ```console - $ docker pull dhi.io/nginx:1.28.0-alpine3.21-dev - ``` - -In the following Dockerfile, the `FROM` instructions use `dhi.io/node:24-alpine3.22-dev` and `dhi.io/nginx:1.28.0-alpine3.21-dev` as the base images. - -```dockerfile -# ========================================= -# Stage 1: Build the React.js Application -# ========================================= - -# Use a lightweight Node.js image for building (customizable via ARG) -FROM dhi.io/node:24-alpine3.22-dev AS builder - -# Set the working directory inside the container -WORKDIR /app - -# Copy package-related files first to leverage Docker's caching mechanism -COPY package.json package-lock.json* ./ - -# Install project dependencies using npm ci (ensures a clean, reproducible install) -RUN --mount=type=cache,target=/root/.npm npm ci - -# Copy the rest of the application source code into the container -COPY . . - -# Build the React.js application (outputs to /app/dist) -RUN npm run build - -# ========================================= -# Stage 2: Prepare Nginx to Serve Static Files -# ========================================= - -FROM dhi.io/nginx:1.28.0-alpine3.21-dev AS runner - -# Copy custom Nginx config -COPY nginx.conf /etc/nginx/nginx.conf - -# Copy the static build output from the build stage to Nginx's default HTML serving directory -COPY --chown=nginx:nginx --from=builder /app/dist /usr/share/nginx/html - -# Use a non-root user for security best practices -USER nginx - -# Expose port 8080 to allow HTTP traffic -# Note: The default NGINX container now listens on port 8080 instead of 80 -EXPOSE 8080 - -# Start Nginx directly with custom config -ENTRYPOINT ["nginx", "-c", "/etc/nginx/nginx.conf"] -CMD ["-g", "daemon off;"] -``` - -{{< /tab >}} -{{< tab name="Using the Docker Official Image" >}} - -Create a file named `Dockerfile` with the following contents: - -```dockerfile -# ========================================= -# Stage 1: Build the React.js Application -# ========================================= -ARG NODE_VERSION=24.12.0-alpine -ARG NGINX_VERSION=alpine3.22 - -# Use a lightweight Node.js image for building (customizable via ARG) -FROM node:${NODE_VERSION} AS builder - -# Set the working directory inside the container -WORKDIR /app - -# Copy package-related files first to leverage Docker's caching mechanism -COPY package.json package-lock.json* ./ - -# Install project dependencies using npm ci (ensures a clean, reproducible install) -RUN --mount=type=cache,target=/root/.npm npm ci - -# Copy the rest of the application source code into the container -COPY . . - -# Build the React.js application (outputs to /app/dist) -RUN npm run build - -# ========================================= -# Stage 2: Prepare Nginx to Serve Static Files -# ========================================= - -FROM nginxinc/nginx-unprivileged:${NGINX_VERSION} AS runner - -# Copy custom Nginx config -COPY nginx.conf /etc/nginx/nginx.conf - -# Copy the static build output from the build stage to Nginx's default HTML serving directory -COPY --chown=nginx:nginx --from=builder /app/dist /usr/share/nginx/html - -# Use a built-in non-root user for security best practices -USER nginx - -# Expose port 8080 to allow HTTP traffic -# Note: The default NGINX container now listens on port 8080 instead of 80 -EXPOSE 8080 - -# Start Nginx directly with custom config -ENTRYPOINT ["nginx", "-c", "/etc/nginx/nginx.conf"] -CMD ["-g", "daemon off;"] -``` - -> [!NOTE] -> We are using nginx-unprivileged instead of the standard NGINX image to follow security best practices. -> Running as a non-root user in the final image: ->- Reduces the attack surface ->- Aligns with Docker’s recommendations for container hardening ->- Helps comply with stricter security policies in production environments - -{{< /tab >}} -{{< /tabs >}} - -### Step 2: Create the compose.yaml file - -Create a file named `compose.yaml` with the following contents: - -```yaml {collapse=true,title=compose.yaml} -services: - server: - build: - context: . - ports: - - 8080:8080 -``` - -### Step 3: Create the .dockerignore file - -The `.dockerignore` file tells Docker which files and folders to exclude when building the image. - - -> [!NOTE] ->This helps: ->- Reduce image size ->- Speed up the build process ->- Prevent sensitive or unnecessary files (like `.env`, `.git`, or `node_modules`) from being added to the final image. -> -> To learn more, visit the [.dockerignore reference](/reference/dockerfile.md#dockerignore-file). - -Create a file named `.dockerignore` with the following contents: - -```dockerignore -# Ignore dependencies and build output -node_modules/ -dist/ -out/ -.tmp/ -.cache/ - -# Ignore Vite, Webpack, and React-specific build artifacts -.vite/ -.vitepress/ -.eslintcache -.npm/ -coverage/ -jest/ -cypress/ -cypress/screenshots/ -cypress/videos/ -reports/ - -# Ignore environment and config files (sensitive data) -*.env* -*.log - -# Ignore TypeScript build artifacts (if using TypeScript) -*.tsbuildinfo - -# Ignore lockfiles (optional if using Docker for package installation) -npm-debug.log* -yarn-debug.log* -yarn-error.log* -pnpm-debug.log* - -# Ignore local development files -.git/ -.gitignore -.vscode/ -.idea/ -*.swp -.DS_Store -Thumbs.db - -# Ignore Docker-related files (to avoid copying unnecessary configs) -Dockerfile -.dockerignore -docker-compose.yml -docker-compose.override.yml - -# Ignore build-specific cache files -*.lock - -``` - -### Step 4: Create the `nginx.conf` file - -To serve your React.js application efficiently inside the container, you’ll configure NGINX with a custom setup. This configuration is optimized for performance, browser caching, gzip compression, and support for client-side routing. - -Create a file named `nginx.conf` in the root of your project directory, and add the following content: - -> [!NOTE] -> To learn more about configuring NGINX, see the [official NGINX documentation](https://nginx.org/en/docs/). - - -```nginx -worker_processes auto; - -# Store PID in /tmp (always writable) -pid /tmp/nginx.pid; - -events { - worker_connections 1024; -} - -http { - include /etc/nginx/mime.types; - default_type application/octet-stream; - - # Disable logging to avoid permission issues - access_log off; - error_log /dev/stderr warn; - - # Optimize static file serving - sendfile on; - tcp_nopush on; - tcp_nodelay on; - keepalive_timeout 65; - keepalive_requests 1000; - - # Gzip compression for optimized delivery - gzip on; - gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript image/svg+xml; - gzip_min_length 256; - gzip_vary on; - - server { - listen 8080; - server_name localhost; - - # Root directory where React.js build files are placed - root /usr/share/nginx/html; - index index.html; - - # Serve React.js static files with proper caching - location / { - try_files $uri /index.html; - } - - # Serve static assets with long cache expiration - location ~* \.(?:ico|css|js|gif|jpe?g|png|woff2?|eot|ttf|svg|map)$ { - expires 1y; - access_log off; - add_header Cache-Control "public, immutable"; - } - - # Handle React.js client-side routing - location /static/ { - expires 1y; - add_header Cache-Control "public, immutable"; - } - } -} -``` - -### Step 5: Build the React.js application image - -With your custom configuration in place, you're now ready to build the Docker image for your React.js application. - -The updated setup includes: - -- Optimized browser caching and gzip compression -- Secure, non-root logging to avoid permission issues -- Support for React client-side routing by redirecting unmatched routes to `index.html` - -After completing the previous steps, your project directory should now contain the following files: - -```text -├── docker-reactjs-sample/ -│ ├── Dockerfile -│ ├── .dockerignore -│ ├── compose.yaml -│ └── nginx.conf -``` - -Now that your Dockerfile is configured, you can build the Docker image for your React.js application. - -> [!NOTE] -> The `docker build` command packages your application into an image using the instructions in the Dockerfile. It includes all necessary files from the current directory (called the [build context](/build/concepts/context/#what-is-a-build-context)). - -Run the following command from the root of your project: - -```console -$ docker build --tag docker-reactjs-sample . -``` - -What this command does: -- Uses the Dockerfile in the current directory (.) -- Packages the application and its dependencies into a Docker image -- Tags the image as docker-reactjs-sample so you can reference it later - - -### Step 6: View local images - -After building your Docker image, you can check which images are available on your local machine using either the Docker CLI or [Docker Desktop](/manuals/desktop/use-desktop/images.md). Since you're already working in the terminal, let's use the Docker CLI. - -To list all locally available Docker images, run the following command: - -```console -$ docker images -``` - -Example Output: - -```shell -REPOSITORY TAG IMAGE ID CREATED SIZE -docker-reactjs-sample latest f39b47a97156 14 seconds ago 75.8MB -``` - -This output provides key details about your images: - -- **Repository** – The name assigned to the image. -- **Tag** – A version label that helps identify different builds (e.g., latest). -- **Image ID** – A unique identifier for the image. -- **Created** – The timestamp indicating when the image was built. -- **Size** – The total disk space used by the image. - -If the build was successful, you should see `docker-reactjs-sample` image listed. - ---- - -## Run the containerized application - -In the previous step, you created a Dockerfile for your React.js application and built a Docker image using the docker build command. Now it’s time to run that image in a container and verify that your application works as expected. - - -Inside the `docker-reactjs-sample` directory, run the following command in a -terminal. - -```console -$ docker compose up --build -``` - -Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see a simple React.js web application. - -Press `ctrl+c` in the terminal to stop your application. - -### Run the application in the background - -You can run the application detached from the terminal by adding the `-d` -option. Inside the `docker-reactjs-sample` directory, run the following command -in a terminal. - -```console -$ docker compose up --build -d -``` - -Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see a simple web application preview. - - -To confirm that the container is running, use `docker ps` command: - -```console -$ docker ps -``` - -This will list all active containers along with their ports, names, and status. Look for a container exposing port 8080. - -Example Output: - -```shell -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -88bced6ade95 docker-reactjs-sample-server "nginx -c /etc/nginx…" About a minute ago Up About a minute 0.0.0.0:8080->8080/tcp docker-reactjs-sample-server-1 -``` - - -To stop the application, run: - -```console -$ docker compose down -``` - - -> [!NOTE] -> For more information about Compose commands, see the [Compose CLI -> reference](/reference/cli/docker/compose/). - ---- - -## Summary - -In this guide, you learned how to containerize, build, and run a React.js application using Docker. By following best practices, you created a secure, optimized, and production-ready setup. - -What you accomplished: -- Created a multi-stage `Dockerfile` that compiles the React.js application and serves the static files using Nginx. -- Created a `.dockerignore` file to exclude unnecessary files and keep the image clean and efficient. -- Built your Docker image using `docker build`. -- Ran the container using `docker compose up`, both in the foreground and in detached mode. -- Verified that the app was running by visiting [http://localhost:8080](http://localhost:8080). -- Learned how to stop the containerized application using `docker compose down`. - -You now have a fully containerized React.js application, running in a Docker container, and ready for deployment across any environment with confidence and consistency. - ---- - -## Related resources - -Explore official references and best practices to sharpen your Docker workflow: - -- [Multi-stage builds](/build/building/multi-stage/) – Learn how to separate build and runtime stages. -- [Best practices for writing Dockerfiles](/develop/develop-images/dockerfile_best-practices/) – Write efficient, maintainable, and secure Dockerfiles. -- [Build context in Docker](/build/concepts/context/) – Learn how context affects image builds. -- [`docker build` CLI reference](/reference/cli/docker/image/build/) – Build Docker images from a Dockerfile. -- [`docker images` CLI reference](/reference/cli/docker/image/ls/) – Manage and inspect local Docker images. -- [`docker compose up` CLI reference](/reference/cli/docker/compose/up/) – Start and run multi-container applications. -- [`docker compose down` CLI reference](/reference/cli/docker/compose/down/) – Stop and remove containers, networks, and volumes. - ---- - -## Next steps - -With your React.js application now containerized, you're ready to move on to the next step. - -In the next section, you'll learn how to develop your application using Docker containers, enabling a consistent, isolated, and reproducible development environment across any machine. - diff --git a/content/guides/reactjs/deploy.md b/content/guides/reactjs/deploy.md deleted file mode 100644 index 37e6a256e0a2..000000000000 --- a/content/guides/reactjs/deploy.md +++ /dev/null @@ -1,194 +0,0 @@ ---- -title: Test your React.js deployment -linkTitle: Test your deployment -weight: 60 -keywords: deploy, kubernetes, react, react.js -description: Learn how to deploy locally to test and debug your Kubernetes deployment - ---- - -## Prerequisites - -Before you begin, make sure you’ve completed the following: -- Complete all the previous sections of this guide, starting with [Containerize React.js application](containerize.md). -- [Enable Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. - -> **New to Kubernetes?** -> Visit the [Kubernetes basics tutorial](https://kubernetes.io/docs/tutorials/kubernetes-basics/) to get familiar with how clusters, pods, deployments, and services work. - ---- - -## Overview - -This section guides you through deploying your containerized React.js application locally using [Docker Desktop’s built-in Kubernetes](/desktop/kubernetes/). Running your app in a local Kubernetes cluster allows you to closely simulate a real production environment, enabling you to test, validate, and debug your workloads with confidence before promoting them to staging or production. - ---- - -## Create a Kubernetes YAML file - -Follow these steps to define your deployment configuration: - -1. In the root of your project, create a new file named: reactjs-sample-kubernetes.yaml - -2. Open the file in your IDE or preferred text editor. - -3. Add the following configuration, and be sure to replace `{DOCKER_USERNAME}` and `{DOCKERHUB_PROJECT_NAME}` with your actual Docker Hub username and repository name from the previous [Automate your builds with GitHub Actions](configure-github-actions.md). - - -```yaml -apiVersion: apps/v1 -kind: Deployment -metadata: - name: reactjs-sample - namespace: default -spec: - replicas: 1 - selector: - matchLabels: - app: reactjs-sample - template: - metadata: - labels: - app: reactjs-sample - spec: - containers: - - name: reactjs-container - image: {DOCKER_USERNAME}/{DOCKERHUB_PROJECT_NAME}:latest - imagePullPolicy: Always - ports: - - containerPort: 8080 ---- -apiVersion: v1 -kind: Service -metadata: - name: reactjs-sample-service - namespace: default -spec: - type: NodePort - selector: - app: reactjs-sample - ports: - - port: 8080 - targetPort: 8080 - nodePort: 30001 -``` - -This manifest defines two key Kubernetes resources, separated by `---`: - -- Deployment - Deploys a single replica of your React.js application inside a pod. The pod uses the Docker image built and pushed by your GitHub Actions CI/CD workflow - (refer to [Automate your builds with GitHub Actions](configure-github-actions.md)). - The container listens on port `8080`, which is typically used by [Nginx](https://nginx.org/en/docs/) to serve your production React app. - -- Service (NodePort) - Exposes the deployed pod to your local machine. - It forwards traffic from port `30001` on your host to port `8080` inside the container. - This lets you access the application in your browser at [http://localhost:30001](http://localhost:30001). - -> [!NOTE] -> To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). - ---- - -## Deploy and check your application - -Follow these steps to deploy your containerized React.js app into a local Kubernetes cluster and verify that it’s running correctly. - -### Step 1. Apply the Kubernetes configuration - -In your terminal, navigate to the directory where your `reactjs-sample-kubernetes.yaml` file is located, then deploy the resources using: - -```console - $ kubectl apply -f reactjs-sample-kubernetes.yaml -``` - -If everything is configured properly, you’ll see confirmation that both the Deployment and the Service were created: - -```shell - deployment.apps/reactjs-sample created - service/reactjs-sample-service created -``` - -This output means that both the Deployment and the Service were successfully created and are now running inside your local cluster. - -### Step 2. Check the deployment status - -Run the following command to check the status of your deployment: - -```console - $ kubectl get deployments -``` - -You should see an output similar to: - -```shell - NAME READY UP-TO-DATE AVAILABLE AGE - reactjs-sample 1/1 1 1 14s -``` - -This confirms that your pod is up and running with one replica available. - -### Step 3. Verify the service exposure - -Check if the NodePort service is exposing your app to your local machine: - -```console -$ kubectl get services -``` - -You should see something like: - -```shell -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -reactjs-sample-service NodePort 10.100.244.65 8080:30001/TCP 1m -``` - -This output confirms that your app is available via NodePort on port 30001. - -### Step 4. Access your app in the browser - -Open your browser and navigate to [http://localhost:30001](http://localhost:30001). - -You should see your production-ready React.js Sample application running — served by your local Kubernetes cluster. - -### Step 5. Clean up Kubernetes resources - -Once you're done testing, you can delete the deployment and service using: - -```console - $ kubectl delete -f reactjs-sample-kubernetes.yaml -``` - -Expected output: - -```shell - deployment.apps "reactjs-sample" deleted - service "reactjs-sample-service" deleted -``` - -This ensures your cluster stays clean and ready for the next deployment. - ---- - -## Summary - -In this section, you learned how to deploy your React.js application to a local Kubernetes cluster using Docker Desktop. This setup allows you to test and debug your containerized app in a production-like environment before deploying it to the cloud. - -What you accomplished: - -- Created a Kubernetes Deployment and NodePort Service for your React.js app -- Used `kubectl apply` to deploy the application locally -- Verified the app was running and accessible at `http://localhost:30001` -- Cleaned up your Kubernetes resources after testing - ---- - -## Related resources - -Explore official references and best practices to sharpen your Kubernetes deployment workflow: - -- [Kubernetes documentation](https://kubernetes.io/docs/home/) – Learn about core concepts, workloads, services, and more. -- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) – Use Docker Desktop’s built-in Kubernetes support for local testing and development. -- [`kubectl` CLI reference](https://kubernetes.io/docs/reference/kubectl/) – Manage Kubernetes clusters from the command line. -- [Kubernetes Deployment resource](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) – Understand how to manage and scale applications using Deployments. -- [Kubernetes Service resource](https://kubernetes.io/docs/concepts/services-networking/service/) – Learn how to expose your application to internal and external traffic. \ No newline at end of file diff --git a/content/guides/reactjs/develop.md b/content/guides/reactjs/develop.md deleted file mode 100644 index 8a8b5430dba6..000000000000 --- a/content/guides/reactjs/develop.md +++ /dev/null @@ -1,205 +0,0 @@ ---- -title: Use containers for React.js development -linkTitle: Develop your app -weight: 30 -keywords: react.js, development, node -description: Learn how to develop your React.js application locally using containers. - ---- - -## Prerequisites - -Complete [Containerize React.js application](containerize.md). - ---- - -## Overview - -In this section, you'll learn how to set up both production and development environments for your containerized React.js application using Docker Compose. This setup allows you to serve a static production build via Nginx and to develop efficiently inside containers using a live-reloading dev server with Compose Watch. - -You’ll learn how to: -- Configure separate containers for production and development -- Enable automatic file syncing using Compose Watch in development -- Debug and live-preview your changes in real-time without manual rebuilds - ---- - -## Automatically update services (development mode) - -Use Compose Watch to automatically sync source file changes into your containerized development environment. This provides a seamless, efficient development experience without needing to restart or rebuild containers manually. - -## Step 1: Create a development Dockerfile - -Create a file named `Dockerfile.dev` in your project root with the following content: - -```dockerfile -# ========================================= -# Stage 1: Develop the React.js Application -# ========================================= -ARG NODE_VERSION=24.12.0-alpine - -# Use a lightweight Node.js image for development -FROM node:${NODE_VERSION} AS dev - -# Set the working directory inside the container -WORKDIR /app - -# Copy package-related files first to leverage Docker's caching mechanism -COPY package.json package-lock.json* ./ - -# Install project dependencies -RUN --mount=type=cache,target=/root/.npm npm install - -# Copy the rest of the application source code into the container -COPY . . - -# Expose the port used by the Vite development server -EXPOSE 5173 - -# Use a default command, can be overridden in Docker compose.yml file -CMD ["npm", "run", "dev"] -``` - -This file sets up a lightweight development environment for your React app using the dev server. - - -### Step 2: Update your `compose.yaml` file - -Open your `compose.yaml` file and define two services: one for production (`react-prod`) and one for development (`react-dev`). - -Here’s an example configuration for a React.js application: - -```yaml -services: - react-prod: - build: - context: . - dockerfile: Dockerfile - image: docker-reactjs-sample - ports: - - "8080:8080" - - react-dev: - build: - context: . - dockerfile: Dockerfile.dev - ports: - - "5173:5173" - develop: - watch: - - action: sync - path: . - target: /app - -``` -- The `react-prod` service builds and serves your static production app using Nginx. -- The `react-dev` service runs your React development server with live reload and hot module replacement. -- `watch` triggers file sync with Compose Watch. - -> [!NOTE] -> For more details, see the official guide: [Use Compose Watch](/manuals/compose/how-tos/file-watch.md). - -### Step 3: Update vite.config.ts to ensure it works properly inside Docker - -To make Vite’s development server work reliably inside Docker, you need to update your vite.config.ts with the correct settings. - -Open the `vite.config.ts` file in your project root and update it as follows: - -```ts -/// - -import { defineConfig } from "vite"; -import react from "@vitejs/plugin-react"; - -export default defineConfig({ - base: "/", - plugins: [react()], - server: { - host: true, - port: 5173, - strictPort: true, - }, -}); -``` - -> [!NOTE] -> The `server` options in `vite.config.ts` are essential for running Vite inside Docker: -> - `host: true` allows the dev server to be accessible from outside the container. -> - `port: 5173` sets a consistent development port (must match the one exposed in Docker). -> - `strictPort: true` ensures Vite fails clearly if the port is unavailable, rather than switching silently. -> -> For full details, refer to the [Vite server configuration docs](https://vitejs.dev/config/server-options.html). - - -After completing the previous steps, your project directory should now contain the following files: - -```text -├── docker-reactjs-sample/ -│ ├── Dockerfile -│ ├── Dockerfile.dev -│ ├── .dockerignore -│ ├── compose.yaml -│ └── nginx.conf -``` - -### Step 4: Start Compose Watch - -Run the following command from your project root to start your container in watch mode: - -```console -$ docker compose watch react-dev -``` - -### Step 5: Test Compose Watch with React - -To verify that Compose Watch is working correctly: - -1. Open the `src/App.tsx` file in your text editor. - -2. Locate the following line: - - ```html -

    Vite + React

    - ``` - -3. Change it to: - - ```html -

    Hello from Docker Compose Watch

    - ``` - -4. Save the file. - -5. Open your browser at [http://localhost:5173](http://localhost:5173). - -You should see the updated text appear instantly, without needing to rebuild the container manually. This confirms that file watching and automatic synchronization are working as expected. - ---- - -## Summary - -In this section, you set up a complete development and production workflow for your React.js application using Docker and Docker Compose. - -Here's what you achieved: -- Created a `Dockerfile.dev` to streamline local development with hot reloading -- Defined separate `react-dev` and `react-prod` services in your `compose.yaml` file -- Enabled real-time file syncing using Compose Watch for a smoother development experience -- Verified that live updates work seamlessly by modifying and previewing a component - -With this setup, you're now equipped to build, run, and iterate on your React.js app entirely within containers—efficiently and consistently across environments. - ---- - -## Related resources - -Deepen your knowledge and improve your containerized development workflow with these guides: - -- [Using Compose Watch](/manuals/compose/how-tos/file-watch.md) – Automatically sync source changes during development -- [Multi-stage builds](/manuals/build/building/multi-stage.md) – Create efficient, production-ready Docker images -- [Dockerfile best practices](/build/building/best-practices/) – Write clean, secure, and optimized Dockerfiles. -- [Compose file reference](/compose/compose-file/) – Learn the full syntax and options available for configuring services in `compose.yaml`. -- [Docker volumes](/storage/volumes/) – Persist and manage data between container runs - -## Next steps - -In the next section, you'll learn how to run unit tests for your React.js application inside Docker containers. This ensures consistent testing across all environments and removes dependencies on local machine setup. diff --git a/content/guides/reactjs/run-tests.md b/content/guides/reactjs/run-tests.md deleted file mode 100644 index dc72025d43f4..000000000000 --- a/content/guides/reactjs/run-tests.md +++ /dev/null @@ -1,179 +0,0 @@ ---- -title: Run React.js tests in a container -linkTitle: Run your tests -weight: 40 -keywords: react.js, react, test, vitest -description: Learn how to run your React.js tests in a container. - ---- - -## Prerequisites - -Complete all the previous sections of this guide, starting with [Containerize React.js application](containerize.md). - -## Overview - -Testing is a critical part of the development process. In this section, you'll learn how to: - -- Run unit tests using Vitest inside a Docker container. -- Use Docker Compose to run tests in an isolated, reproducible environment. - -You’ll use [Vitest](https://vitest.dev) — a blazing fast test runner designed for Vite — along with [Testing Library](https://testing-library.com/) for assertions. - ---- - -## Run tests during development - -`docker-reactjs-sample` application includes a sample test file at location: - -```console -$ src/App.test.tsx -``` - -This file uses Vitest and React Testing Library to verify the behavior of `App` component. - -### Step 1: Install Vitest and React Testing Library - -If you haven’t already added the necessary testing tools, install them by running: - -```console -$ npm install --save-dev vitest @testing-library/react @testing-library/jest-dom jsdom -``` - -Then, update the scripts section of your `package.json` file to include the following: - -```json -"scripts": { - "test": "vitest run" -} -``` - ---- - -### Step 2: Configure Vitest - -Update `vitest.config.ts` file in your project root with the following configuration: - -```ts {hl_lines="14-18",linenos=true} -/// - -import { defineConfig } from "vite"; -import react from "@vitejs/plugin-react"; - -export default defineConfig({ - base: "/", - plugins: [react()], - server: { - host: true, - port: 5173, - strictPort: true, - }, - test: { - environment: "jsdom", - setupFiles: "./src/setupTests.ts", - globals: true, - }, -}); -``` - -> [!NOTE] -> The `test` options in `vitest.config.ts` are essential for reliable testing inside Docker: -> - `environment: "jsdom"` simulates a browser-like environment for rendering and DOM interactions. -> - `setupFiles: "./src/setupTests.ts"` loads global configuration or mocks before each test file (optional but recommended). -> - `globals: true` enables global test functions like `describe`, `it`, and `expect` without importing them. -> -> For more details, see the official [Vitest configuration docs](https://vitest.dev/config/). - -### Step 3: Update compose.yaml - -Add a new service named `react-test` to your `compose.yaml` file. This service allows you to run your test suite in an isolated containerized environment. - -```yaml {hl_lines="22-26",linenos=true} -services: - react-dev: - build: - context: . - dockerfile: Dockerfile.dev - ports: - - "5173:5173" - develop: - watch: - - action: sync - path: . - target: /app - - react-prod: - build: - context: . - dockerfile: Dockerfile - image: docker-reactjs-sample - ports: - - "8080:8080" - - react-test: - build: - context: . - dockerfile: Dockerfile.dev - command: ["npm", "run", "test"] - -``` - -The react-test service reuses the same `Dockerfile.dev` used for [development](develop.md) and overrides the default command to run tests with `npm run test`. This setup ensures a consistent test environment that matches your local development configuration. - - -After completing the previous steps, your project directory should contain the following files: - -```text -├── docker-reactjs-sample/ -│ ├── Dockerfile -│ ├── Dockerfile.dev -│ ├── .dockerignore -│ ├── compose.yaml -│ └── nginx.conf -``` - -### Step 4: Run the tests - -To execute your test suite inside the container, run the following command from your project root: - -```console -$ docker compose run --rm react-test -``` - -This command will: -- Start the `react-test` service defined in your `compose.yaml` file. -- Execute the `npm run test` script using the same environment as development. -- Automatically remove the container after the tests complete [`docker compose run --rm`](/reference/cli/docker/compose/run/) command. - -> [!NOTE] -> For more information about Compose commands, see the [Compose CLI -> reference](/reference/cli/docker/compose/). - ---- - -## Summary - -In this section, you learned how to run unit tests for your React.js application inside a Docker container using Vitest and Docker Compose. - -What you accomplished: -- Installed and configured Vitest and React Testing Library for testing React components. -- Created a `react-test` service in `compose.yaml` to isolate test execution. -- Reused the development `Dockerfile.dev` to ensure consistency between dev and test environments. -- Ran tests inside the container using `docker compose run --rm react-test`. -- Ensured reliable, repeatable testing across environments without relying on local machine setup. - ---- - -## Related resources - -Explore official references and best practices to sharpen your Docker testing workflow: - -- [Dockerfile reference](/reference/dockerfile/) – Understand all Dockerfile instructions and syntax. -- [Best practices for writing Dockerfiles](/develop/develop-images/dockerfile_best-practices/) – Write efficient, maintainable, and secure Dockerfiles. -- [Compose file reference](/compose/compose-file/) – Learn the full syntax and options available for configuring services in `compose.yaml`. -- [`docker compose run` CLI reference](/reference/cli/docker/compose/run/) – Run one-off commands in a service container. ---- - -## Next steps - -Next, you’ll learn how to set up a CI/CD pipeline using GitHub Actions to automatically build and test your React.js application in a containerized environment. This ensures your code is validated on every push or pull request, maintaining consistency and reliability across your development workflow. diff --git a/content/guides/ros2/_index.md b/content/guides/ros2/_index.md index f40208c712a6..415969b1f4e7 100644 --- a/content/guides/ros2/_index.md +++ b/content/guides/ros2/_index.md @@ -5,16 +5,18 @@ description: Learn how to containerize and develop ROS 2 applications using Dock keywords: ros2, robotics, devcontainers, python, cpp, Dockerfile, rviz summary: | This guide details how to containerize ROS 2 applications using Docker. -toc_min: 1 -toc_max: 2 -languages: [python] -tags: [frameworks] +aliases: + - /guides/ros2/develop/ + - /guides/ros2/run-ros2/ + - /guides/ros2/turtlesim-example/ params: + tags: [deployment] time: 30 minutes image: /images/guides/ros2.jpg featured: true --- + > **Acknowledgment** > > This guide is a community contribution. Docker would like to thank @@ -46,3 +48,415 @@ Before you begin, make sure you're familiar with the following: ## What's next? Start by setting up your ROS 2 development environment using Docker and dev containers. + +## Run ROS 2 in a container + +### Overview + +In this section, you will run ROS 2 in an isolated Docker container using official ROS 2 images, verify that ROS 2 is working, and install additional ROS 2 packages for development and testing. + +--- + +### Run ROS 2 in a container + +The fastest way to get started with ROS 2 is to use the [official Docker image](https://hub.docker.com/_/ros/). To pull an image, start a container, and open an interactive bash shell: + +1. Pull and run the official ROS 2 Docker image: + + ```console + $ docker run -it ros:humble + ``` + + This guide uses the Humble distribution. You can replace `humble` with another supported distribution such as `rolling`, `jazzy`, or `iron`. + + > [!NOTE] + > + > This environment is temporary and does not maintain persistence. + > Any files you create or packages you install will be deleted once the container is stopped or removed. + +2. Verify ROS 2 is working: + + ```console + $ echo $ROS_DISTRO + ``` + + You should see output similar to: + + ```text + humble + ``` + +### Install ROS 2 packages + +The official ROS 2 images include core packages. To install additional packages, use the `apt` package manager: + +1. Update the package manager: + + ```console + $ sudo apt update + ``` + +2. Install the desired package: + + ```console + $ sudo apt install $PACKAGE_NAME + ``` + +Replace `$PACKAGE_NAME` with any package you want to install. + +Some commonly used packages include: + +- `ros-humble-turtlesim` - Visualization and simulation tool +- `ros-humble-rviz2` - 3D visualization tool +- `ros-humble-rqt` - Qt-based ROS graphical tools +- `ros-humble-demo-nodes-cpp` - C++ demo nodes +- `ros-humble-demo-nodes-py` - Python demo nodes +- `ros-humble-colcon-common-extensions` - Build system extensions + +### Summary + +In this section, you pulled an official ROS 2 Docker image, launched an interactive session, and extended the container's capabilities by installing additional ROS 2 packages using apt. + +### Next steps + +In the next section, you will configure a persistent workspace to ensure your code and modifications are saved across sessions. + +## Build and develop a ROS 2 workspace + +### Overview + +In this section, you will set up a ROS 2 workspace using Docker and development containers, review the workspace layout, open the workspace in Visual Studio Code, and edit and build ROS 2 projects inside the container. + +--- + +### Get the sample ROS 2 workspace + +A consistent workspace simplifies managing ROS 2 projects and build artifacts across different distributions. + +1. Open a terminal and clone the sample workspace repository: + + ```console + $ git clone https://github.com/shakirth-anisha/docker-ros2-workspace.git + $ cd docker-ros2-workspace + + ``` + + Moving forward, Linux users can use the `ws_linux` folder, and macOS users can use `ws_mac`. + +2. Verify the workspace structure: + + ```text + ws_linux/ + ├── compose.yml + ├── Dockerfile + └── src/ + ├── package1/ + └── package2/ + + ws_mac/ + ├── compose.yml + ├── Dockerfile + └── src/ + ├── package1/ + └── package2/ + + ``` + +3. Explore the workspace layout + +- `compose.yml` : Defines how Docker Compose builds and runs the ROS 2 container, including mounts, environment variables, and networking settings. +- `Dockerfile` : Builds the ROS 2 development image. It uses an official ROS 2 base image, creates a non-root development user, and installs required system and ROS 2 dependencies. +- `src` : Contains all ROS 2 packages. This directory is mounted into the container as the active workspace. + +### Open and build the container + +1. Execute the following commands to build and start the container: + + For Linux: + + ```console + $ cd ws_linux + $ docker compose up -d + $ docker compose exec ros2 /bin/bash + ``` + + For macOS: + + ```console + $ cd ws_mac + $ docker compose up -d + $ docker compose exec ros2 /bin/bash + ``` + + This command builds the Docker image defined in your `Dockerfile` and starts the container in the background. + + > [!NOTE] + > + > Building the image may take several minutes during the first run + > as the CLI pulls the base ROS 2 image and installs required dependencies. + > Subsequent starts will be significantly faster. + +2. Once the container is running, execute commands inside it using `exec`: + + ```console + $ docker compose exec ros2 /bin/bash + ``` + +3. Inside the container terminal, verify the environment: + +```console +$ echo $ROS_VERSION +$ which colcon +``` + +All commands should execute successfully inside the container. + +### Switch ROS 2 distributions + +Update the base image in your `Dockerfile`, changing from `humble` to another distribution like `rolling`, `jazzy`, or `iron`. + +### Summary + +In this section, you learned how to create a structured workspace, write a Dockerfile with development tools, and configure a Docker Compose setup. Your ROS 2 development environment is now ready with a consistent, reproducible setup across any machine. + +### Next steps + +In the next section, you'll run a complete end-to-end example with Turtlesim. + +## Run a complete example with Turtlesim + +### Overview + +Turtlesim is a simple simulation tool that demonstrates fundamental ROS 2 concepts such as nodes, topics, and services. In this section, you'll run a complete example with Turtlesim, control the turtle, monitor topics, and visualize the system with rqt. + +--- + +### Configure display forwarding + +#### Linux + +Allow Docker access to your X server: + +```console +$ xhost +local:docker +``` + +#### macOS + +On macOS, use XQuartz to provide X11 support. Install XQuartz using Homebrew: + +1. Install XQuartz using Homebrew: + + ```console + $ brew install --cask xquartz + ``` + +2. Open XQuartz from Applications, then navigate to `Preferences > Security` and enable `Allow connections from network clients`. Restart your computer to ensure the changes take effect. + +3. After rebooting, open a terminal and allow local connections: + + ```console + $ defaults write org.xquartz.X11 nolisten_tcp -bool false + $ xhost +localhost + $ xhost + 127.0.0.1 + ``` + +### Start the container + +Start the container using the same Docker Compose setup from the workspace section. + +For Linux: + +```console +$ cd ws_linux +$ docker compose up -d +$ docker compose exec ros2 /bin/bash +``` + +For macOS: + +```console +$ cd ws_mac +$ docker compose up -d +$ docker compose exec ros2 /bin/bash +``` + +### Install and run Turtlesim + +Inside the container, install the Turtlesim package: + +1. Update the package manager: + + ```console + $ sudo apt update + ``` + +2. Install the Turtlesim package: + + ```console + $ sudo apt install -y ros-humble-turtlesim + ``` + +3. Run the Turtlesim node: + + ```console + $ ros2 run turtlesim turtlesim_node + ``` + +A window should appear on your desktop showing a turtle in a grid. + +### Control the turtle + +1. Open a new terminal and connect to the same container, then start the keyboard teleop node: + + ```console + $ ros2 run turtlesim turtle_teleop_key + ``` + + This node allows you to control the turtle using your keyboard. Use the arrow keys to move the turtle forward, backward, left, and right. Press `Ctrl+C` to stop the teleop node. + +2. Move the turtle around the window. You should see it draw a path as it moves. + +### Monitor topics + +1. Open another terminal and connect to the same container, then list all active topics: + + ```console + $ ros2 topic list + ``` + + You should see output similar to the following: + + ```text + /parameter_events + /rosout + /turtle1/cmd_vel + /turtle1/color_sensor + /turtle1/pose + ``` + +2. Get information about a specific topic: + + ```console + $ ros2 topic info /turtle1/pose + ``` + + You'll see the topic type and which nodes publish and subscribe to it. + +### Visualize the system with rqt + +1. Open another terminal and connect to the same container, then update the package manager: + + ```console + $ sudo apt update + ``` + +2. Install rqt: + + ```console + $ sudo apt install -y 'ros-humble-rqt*' + ``` + +3. Start rqt: + + ```console + $ ros2 run rqt_gui rqt_gui + ``` + +An rqt window should appear. rqt provides several useful plugins for visualizing and monitoring ROS 2 systems. + +#### Node Graph + +You can explore the node graph by navigating to **Plugins > Introspection > Node Graph**. A new tab opens showing nodes and topics with connections illustrated as lines. This visualization demonstrates how the teleop node sends velocity commands to the Turtlesim node, and how the Turtlesim node publishes position data back through topics. + +#### Topic Monitor + +You can monitor active topics by navigating to **Plugins > Topics > Topic Monitor**. A new tab opens displaying all active topics and their current values. Select the eye icon next to `/turtle1/pose` to monitor it. As you move the turtle, watch the pose values update in real time, showing the position of the turtle and orientation changing based on your commands. + +#### Service Caller + +You can call services from rqt using **Plugins > Services > Service Caller**. Select a service such as `/turtle1/teleport_absolute`, enter values for the request fields, and select **Call** to send the request. + +#### Plots + +To plot topic data over time navigate to **Plugins > Visualization > Plot**. For example, in the Plot window, type `/turtle1/pose/x` in the Topic field and press Enter. Move the turtle and watch the X position displayed as a graph over time. + +### Call ROS 2 services + +Turtlesim provides services for actions such as repositioning the turtle and clearing the path. + +1. List available services: + + ```console + $ ros2 service list + ``` + + You should see services such as `/turtle1/set_pen` (to change pen color and width), `/turtle1/teleport_absolute` (to move the turtle to a specific position), and `/turtle1/teleport_relative` (to move the turtle relative to its current position). + +2. Teleport the turtle to a new position: + + ```console + $ ros2 service call /turtle1/teleport_absolute turtlesim/srv/TeleportAbsolute " + x: 1.0 + y: 3.0 + theta: 0.0 + " + ``` + + The turtle should instantly move to the specified position (1.0, 3.0). + +### Create a simple publisher + +1. Create a Python script that publishes velocity commands to control the turtle programmatically. In a new terminal, create a file called `move_turtle.py`: + + ```python + import rclpy + from geometry_msgs.msg import Twist + import time + + def main(): + rclpy.init() + node = rclpy.create_node('turtle_mover') + publisher = node.create_publisher(Twist, 'turtle1/cmd_vel', 10) + + # Create a twist message + msg = Twist() + msg.linear.x = 2.0 # Move forward at 2 m/s + msg.angular.z = 1.0 # Rotate at 1 rad/s + + # Publish the message + for i in range(50): + publisher.publish(msg) + time.sleep(0.1) + + # Stop the turtle + msg.linear.x = 0.0 + msg.angular.z = 0.0 + publisher.publish(msg) + + node.destroy_node() + rclpy.shutdown() + + if __name__ == '__main__': + main() + ``` + +2. Run the script: + + ```console + $ python3 move_turtle.py + ``` + + The turtle should move in a circular motion for 5 seconds and then stop. + +### Summary + +In this section, you configured display forwarding, used the Turtlesim nodes, inspected nodes and topics, and visualized the system using rqt. Finally, you interacted with ROS 2 services and created a simple publisher to move the turtle programmatically. + +These fundamental concepts apply directly to real-world robotics applications with actual sensors and actuators. + +### Related resources + +- [ROS 2 Turtlesim tutorials](https://docs.ros.org/en/humble/Tutorials/Beginner-CLI-Tools/Understanding-ROS2-Topics/Understanding-ROS2-Topics.html) +- [ROS 2 Concepts](https://docs.ros.org/en/humble/Concepts.html) +- [Geometry Messages](https://github.com/ros2/geometry2/tree/humble/geometry_msgs) diff --git a/content/guides/ros2/develop.md b/content/guides/ros2/develop.md deleted file mode 100644 index 23c663cedb70..000000000000 --- a/content/guides/ros2/develop.md +++ /dev/null @@ -1,107 +0,0 @@ ---- -title: Build and develop a ROS 2 workspace -linkTitle: Set Up ROS 2 workspace -weight: 15 -keywords: ros2, robotics, docker, dockerfile, devcontainer, vscode, workspace -description: Learn how to develop ROS 2 applications using a Docker based workspace and development containers. ---- - -## Overview - -In this section, you will set up a ROS 2 workspace using Docker and development containers, review the workspace layout, open the workspace in Visual Studio Code, and edit and build ROS 2 projects inside the container. - ---- - -## Get the sample ROS 2 workspace - -A consistent workspace simplifies managing ROS 2 projects and build artifacts across different distributions. - -1. Open a terminal and clone the sample workspace repository: - - ```console - $ git clone https://github.com/shakirth-anisha/docker-ros2-workspace.git - $ cd docker-ros2-workspace - - ``` - - Moving forward, Linux users can use the `ws_linux` folder, and macOS users can use `ws_mac`. - -2. Verify the workspace structure: - - ```text - ws_linux/ - ├── compose.yml - ├── Dockerfile - └── src/ - ├── package1/ - └── package2/ - - ws_mac/ - ├── compose.yml - ├── Dockerfile - └── src/ - ├── package1/ - └── package2/ - - ``` - -3. Explore the workspace layout - -- `compose.yml` : Defines how Docker Compose builds and runs the ROS 2 container, including mounts, environment variables, and networking settings. -- `Dockerfile` : Builds the ROS 2 development image. It uses an official ROS 2 base image, creates a non-root development user, and installs required system and ROS 2 dependencies. -- `src` : Contains all ROS 2 packages. This directory is mounted into the container as the active workspace. - -## Open and build the container - -1. Execute the following commands to build and start the container: - - For Linux: - - ```console - $ cd ws_linux - $ docker compose up -d - $ docker compose exec ros2 /bin/bash - ``` - - For macOS: - - ```console - $ cd ws_mac - $ docker compose up -d - $ docker compose exec ros2 /bin/bash - ``` - - This command builds the Docker image defined in your `Dockerfile` and starts the container in the background. - - > [!NOTE] - > - > Building the image may take several minutes during the first run - > as the CLI pulls the base ROS 2 image and installs required dependencies. - > Subsequent starts will be significantly faster. - -2. Once the container is running, execute commands inside it using `exec`: - - ```console - $ docker compose exec ros2 /bin/bash - ``` - -3. Inside the container terminal, verify the environment: - -```console -$ echo $ROS_VERSION -$ which colcon -``` - -All commands should execute successfully inside the container. - -## Switch ROS 2 distributions - -Update the base image in your `Dockerfile`, changing from `humble` to another distribution like `rolling`, `jazzy`, or `iron`. - -## Summary - -In this section, you learned how to create a structured workspace, write a Dockerfile with development tools, and configure a Docker Compose setup. Your ROS 2 development environment is now ready with a consistent, reproducible setup across any machine. - -## Next steps - -In the next section, you'll run a complete end-to-end example with Turtlesim. diff --git a/content/guides/ros2/run-ros2.md b/content/guides/ros2/run-ros2.md deleted file mode 100644 index 60313d819f24..000000000000 --- a/content/guides/ros2/run-ros2.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -title: Run ROS 2 in a container -linkTitle: Run ROS 2 -weight: 10 -keywords: ros2, robotics, docker, dockerfile, devcontainer, vscode, workspace -description: Run ROS 2 in an isolated Docker container using official ROS 2 images and install additional ROS 2 packages. ---- - -## Overview - -In this section, you will run ROS 2 in an isolated Docker container using official ROS 2 images, verify that ROS 2 is working, and install additional ROS 2 packages for development and testing. - ---- - -## Run ROS 2 in a container - -The fastest way to get started with ROS 2 is to use the [official Docker image](https://hub.docker.com/_/ros/). To pull an image, start a container, and open an interactive bash shell: - -1. Pull and run the official ROS 2 Docker image: - - ```console - $ docker run -it ros:humble - ``` - - This guide uses the Humble distribution. You can replace `humble` with another supported distribution such as `rolling`, `jazzy`, or `iron`. - - > [!NOTE] - > - > This environment is temporary and does not maintain persistence. - > Any files you create or packages you install will be deleted once the container is stopped or removed. - -2. Verify ROS 2 is working: - - ```console - $ echo $ROS_DISTRO - ``` - - You should see output similar to: - - ```text - humble - ``` - -## Install ROS 2 packages - -The official ROS 2 images include core packages. To install additional packages, use the `apt` package manager: - -1. Update the package manager: - - ```console - $ sudo apt update - ``` - -2. Install the desired package: - - ```console - $ sudo apt install $PACKAGE_NAME - ``` - -Replace `$PACKAGE_NAME` with any package you want to install. - -Some commonly used packages include: - -- `ros-humble-turtlesim` - Visualization and simulation tool -- `ros-humble-rviz2` - 3D visualization tool -- `ros-humble-rqt` - Qt-based ROS graphical tools -- `ros-humble-demo-nodes-cpp` - C++ demo nodes -- `ros-humble-demo-nodes-py` - Python demo nodes -- `ros-humble-colcon-common-extensions` - Build system extensions - -## Summary - -In this section, you pulled an official ROS 2 Docker image, launched an interactive session, and extended the container's capabilities by installing additional ROS 2 packages using apt. - -## Next steps - -In the next section, you will configure a persistent workspace to ensure your code and modifications are saved across sessions. diff --git a/content/guides/ros2/turtlesim-example.md b/content/guides/ros2/turtlesim-example.md deleted file mode 100644 index 65abce3f7b98..000000000000 --- a/content/guides/ros2/turtlesim-example.md +++ /dev/null @@ -1,243 +0,0 @@ ---- -title: Run a complete example with Turtlesim -linkTitle: Turtlesim example -weight: 20 -keywords: ros2, turtlesim, example, nodes, topics, teleop -description: Run a complete end-to-end ROS 2 example with Turtlesim. ---- - -## Overview - -Turtlesim is a simple simulation tool that demonstrates fundamental ROS 2 concepts such as nodes, topics, and services. In this section, you'll run a complete example with Turtlesim, control the turtle, monitor topics, and visualize the system with rqt. - ---- - -## Configure display forwarding - -### Linux - -Allow Docker access to your X server: - -```console -$ xhost +local:docker -``` - -### macOS - -On macOS, use XQuartz to provide X11 support. Install XQuartz using Homebrew: - -1. Install XQuartz using Homebrew: - - ```console - $ brew install --cask xquartz - ``` - -2. Open XQuartz from Applications, then navigate to `Preferences > Security` and enable `Allow connections from network clients`. Restart your computer to ensure the changes take effect. - -3. After rebooting, open a terminal and allow local connections: - - ```console - $ defaults write org.xquartz.X11 nolisten_tcp -bool false - $ xhost +localhost - $ xhost + 127.0.0.1 - ``` - -## Start the container - -Start the container using the same Docker Compose setup from the workspace section. - -For Linux: - -```console -$ cd ws_linux -$ docker compose up -d -$ docker compose exec ros2 /bin/bash -``` - -For macOS: - -```console -$ cd ws_mac -$ docker compose up -d -$ docker compose exec ros2 /bin/bash -``` - -## Install and run Turtlesim - -Inside the container, install the Turtlesim package: - -1. Update the package manager: - - ```console - $ sudo apt update - ``` - -2. Install the Turtlesim package: - - ```console - $ sudo apt install -y ros-humble-turtlesim - ``` - -3. Run the Turtlesim node: - - ```console - $ ros2 run turtlesim turtlesim_node - ``` - -A window should appear on your desktop showing a turtle in a grid. - -## Control the turtle - -1. Open a new terminal and connect to the same container, then start the keyboard teleop node: - - ```console - $ ros2 run turtlesim turtle_teleop_key - ``` - - This node allows you to control the turtle using your keyboard. Use the arrow keys to move the turtle forward, backward, left, and right. Press `Ctrl+C` to stop the teleop node. - -2. Move the turtle around the window. You should see it draw a path as it moves. - -## Monitor topics - -1. Open another terminal and connect to the same container, then list all active topics: - - ```console - $ ros2 topic list - ``` - - You should see output similar to the following: - - ```text - /parameter_events - /rosout - /turtle1/cmd_vel - /turtle1/color_sensor - /turtle1/pose - ``` - -2. Get information about a specific topic: - - ```console - $ ros2 topic info /turtle1/pose - ``` - - You'll see the topic type and which nodes publish and subscribe to it. - -## Visualize the system with rqt - -1. Open another terminal and connect to the same container, then update the package manager: - - ```console - $ sudo apt update - ``` - -2. Install rqt: - - ```console - $ sudo apt install -y 'ros-humble-rqt*' - ``` - -3. Start rqt: - - ```console - $ ros2 run rqt_gui rqt_gui - ``` - -An rqt window should appear. rqt provides several useful plugins for visualizing and monitoring ROS 2 systems. - -### Node Graph - -You can explore the node graph by navigating to **Plugins > Introspection > Node Graph**. A new tab opens showing nodes and topics with connections illustrated as lines. This visualization demonstrates how the teleop node sends velocity commands to the Turtlesim node, and how the Turtlesim node publishes position data back through topics. - -### Topic Monitor - -You can monitor active topics by navigating to **Plugins > Topics > Topic Monitor**. A new tab opens displaying all active topics and their current values. Select the eye icon next to `/turtle1/pose` to monitor it. As you move the turtle, watch the pose values update in real time, showing the position of the turtle and orientation changing based on your commands. - -### Service Caller - -You can call services from rqt using **Plugins > Services > Service Caller**. Select a service such as `/turtle1/teleport_absolute`, enter values for the request fields, and select **Call** to send the request. - -### Plots - -To plot topic data over time navigate to **Plugins > Visualization > Plot**. For example, in the Plot window, type `/turtle1/pose/x` in the Topic field and press Enter. Move the turtle and watch the X position displayed as a graph over time. - -## Call ROS 2 services - -Turtlesim provides services for actions such as repositioning the turtle and clearing the path. - -1. List available services: - - ```console - $ ros2 service list - ``` - - You should see services such as `/turtle1/set_pen` (to change pen color and width), `/turtle1/teleport_absolute` (to move the turtle to a specific position), and `/turtle1/teleport_relative` (to move the turtle relative to its current position). - -2. Teleport the turtle to a new position: - - ```console - $ ros2 service call /turtle1/teleport_absolute turtlesim/srv/TeleportAbsolute " - x: 1.0 - y: 3.0 - theta: 0.0 - " - ``` - - The turtle should instantly move to the specified position (1.0, 3.0). - -## Create a simple publisher - -1. Create a Python script that publishes velocity commands to control the turtle programmatically. In a new terminal, create a file called `move_turtle.py`: - - ```python - import rclpy - from geometry_msgs.msg import Twist - import time - - def main(): - rclpy.init() - node = rclpy.create_node('turtle_mover') - publisher = node.create_publisher(Twist, 'turtle1/cmd_vel', 10) - - # Create a twist message - msg = Twist() - msg.linear.x = 2.0 # Move forward at 2 m/s - msg.angular.z = 1.0 # Rotate at 1 rad/s - - # Publish the message - for i in range(50): - publisher.publish(msg) - time.sleep(0.1) - - # Stop the turtle - msg.linear.x = 0.0 - msg.angular.z = 0.0 - publisher.publish(msg) - - node.destroy_node() - rclpy.shutdown() - - if __name__ == '__main__': - main() - ``` - -2. Run the script: - - ```console - $ python3 move_turtle.py - ``` - - The turtle should move in a circular motion for 5 seconds and then stop. - -## Summary - -In this section, you configured display forwarding, used the Turtlesim nodes, inspected nodes and topics, and visualized the system using rqt. Finally, you interacted with ROS 2 services and created a simple publisher to move the turtle programmatically. - -These fundamental concepts apply directly to real-world robotics applications with actual sensors and actuators. - -## Related resources - -- [ROS 2 Turtlesim tutorials](https://docs.ros.org/en/humble/Tutorials/Beginner-CLI-Tools/Understanding-ROS2-Topics/Understanding-ROS2-Topics.html) -- [ROS 2 Concepts](https://docs.ros.org/en/humble/Concepts.html) -- [Geometry Messages](https://github.com/ros2/geometry2/tree/humble/geometry_msgs) diff --git a/content/guides/ruby/_index.md b/content/guides/ruby/_index.md index 5652dc7fe5b2..58fad9d67b5d 100644 --- a/content/guides/ruby/_index.md +++ b/content/guides/ruby/_index.md @@ -6,17 +6,26 @@ keywords: Docker, getting started, ruby, language summary: | This guide explains how to containerize Ruby on Rails applications using Docker. -toc_min: 1 -toc_max: 2 aliases: - /language/ruby/ - /guides/language/ruby/ -languages: [ruby] -tags: [frameworks] + - /language/ruby/build-images/ + - /language/ruby/run-containers/ + - /language/ruby/containerize/ + - /language/ruby/configure-ci-cd/ + - /guides/language/ruby/configure-ci-cd/ + - /language/ruby/develop/ + - /language/ruby/deploy/ + - /guides/ruby/configure-github-actions/ + - /guides/ruby/containerize/ + - /guides/ruby/deploy/ + - /guides/ruby/develop/ params: + tags: [cicd] time: 20 minutes --- + The Ruby language-specific guide teaches you how to containerize a Ruby on Rails application using Docker. In this guide, you’ll learn how to: - Containerize and run a Ruby on Rails application @@ -25,3 +34,839 @@ The Ruby language-specific guide teaches you how to containerize a Ruby on Rails - Deploy your containerized Ruby on Rails application locally to Kubernetes to test and debug your deployment Start by containerizing an existing Ruby on Rails application. + +## Containerize a Ruby on Rails application + +### Prerequisites + +- You have installed the latest version of [Docker Desktop](/get-started/get-docker.md). +- You have a [Git client](https://git-scm.com/downloads). The examples in this section show the Git CLI, but you can use any client. + +### Overview + +This section walks you through containerizing and running a [Ruby on Rails](https://rubyonrails.org/) application. + +Starting from Rails 7.1 [Docker is supported out of the box](https://guides.rubyonrails.org/7_1_release_notes.html#generate-dockerfiles-for-new-rails-applications). This means that you will get a `Dockerfile`, `.dockerignore` and `bin/docker-entrypoint` files generated for you when you create a new Rails application. + +If you have an existing Rails application, you will need to create the Docker assets manually from the examples below. + +### 1. Create Docker assets + +> [!TIP] +> +> [Gordon](/ai/gordon/), Docker's AI assistant, can generate Docker assets for your project. Ask Gordon to create a Dockerfile, Compose file, and `.dockerignore` tailored to your application. + +Rails 7.1 and newer generates multistage Dockerfile out of the box. Following are two versions of such a file: one using Docker Hardened Images (DHIs) and another using the Docker Official Image (DOIs). Although the Dockerfile is generated automatically, understanding its purpose and functionality is important. Reviewing the following example is highly recommended. + +[Docker Hardened Images (DHIs)](https://docs.docker.com/dhi/) are minimal, secure, and production-ready container base and application images maintained by Docker. DHIs are recommended whenever it is possible for better security. They are designed to reduce vulnerabilities and simplify compliance, freely available to everyone with no subscription required, no usage restrictions, and no vendor lock-in. + +Multistage Dockerfiles help create smaller, more efficient images by separating build and runtime dependencies, ensuring only necessary components are included in the final image. Read more in the [Multi-stage builds guide](/get-started/docker-concepts/building-images/multi-stage-builds/). + + + +{{< tabs >}} +{{< tab name="Using DHIs" >}} + +You must authenticate to `dhi.io` before you can pull Docker Hardened Images. Run `docker login dhi.io` to authenticate. + +```dockerfile {title=Dockerfile} +# syntax=docker/dockerfile:1 +# check=error=true + +# This Dockerfile is designed for production, not development. +# docker build -t app . +# docker run -d -p 80:80 -e RAILS_MASTER_KEY= --name app app + +# For a containerized dev environment, see Dev Containers: https://guides.rubyonrails.org/getting_started_with_devcontainer.html + +# Make sure RUBY_VERSION matches the Ruby version in .ruby-version +ARG RUBY_VERSION=3.4.8 +FROM dhi.io/ruby:$RUBY_VERSION-dev AS base + +# Rails app lives here +WORKDIR /rails + +# Install base packages +# Replace libpq-dev with sqlite3 if using SQLite, or libmysqlclient-dev if using MySQL +RUN apt-get update -qq && \ + apt-get install --no-install-recommends -y curl libjemalloc2 libvips libpq-dev && \ + rm -rf /var/lib/apt/lists /var/cache/apt/archives + +# Set production environment +ENV RAILS_ENV="production" \ + BUNDLE_DEPLOYMENT="1" \ + BUNDLE_PATH="/usr/local/bundle" \ + BUNDLE_WITHOUT="development" + +# Throw-away build stage to reduce size of final image +FROM base AS build + +# Install packages needed to build gems +RUN apt-get update -qq && \ + apt-get install --no-install-recommends -y build-essential curl git pkg-config libyaml-dev && \ + rm -rf /var/lib/apt/lists /var/cache/apt/archives + +# Install JavaScript dependencies and Node.js for asset compilation +# +# Uncomment the following lines if you are using NodeJS need to compile assets +# +# ARG NODE_VERSION=18.12.0 +# ARG YARN_VERSION=1.22.19 +# ENV PATH=/usr/local/node/bin:$PATH +# RUN curl -sL https://github.com/nodenv/node-build/archive/master.tar.gz | tar xz -C /tmp/ && \ +# /tmp/node-build-master/bin/node-build "${NODE_VERSION}" /usr/local/node && \ +# npm install -g yarn@$YARN_VERSION && \ +# npm install -g mjml && \ +# rm -rf /tmp/node-build-master + +# Install application gems +COPY Gemfile Gemfile.lock ./ +RUN bundle install && \ + rm -rf ~/.bundle/ "${BUNDLE_PATH}"/ruby/*/cache "${BUNDLE_PATH}"/ruby/*/bundler/gems/*/.git && \ + bundle exec bootsnap precompile --gemfile + +# Install node modules +# +# Uncomment the following lines if you are using NodeJS need to compile assets +# +# COPY package.json yarn.lock ./ +# RUN --mount=type=cache,id=yarn,target=/rails/.cache/yarn YARN_CACHE_FOLDER=/rails/.cache/yarn \ +# yarn install --frozen-lockfile + +# Copy application code +COPY . . + +# Precompile bootsnap code for faster boot times +RUN bundle exec bootsnap precompile app/ lib/ + +# Precompiling assets for production without requiring secret RAILS_MASTER_KEY +RUN SECRET_KEY_BASE_DUMMY=1 ./bin/rails assets:precompile + +# Final stage for app image +FROM base + +# Copy built artifacts: gems, application +COPY --from=build "${BUNDLE_PATH}" "${BUNDLE_PATH}" +COPY --from=build /rails /rails + +# Run and own only the runtime files as a non-root user for security +RUN groupadd --system --gid 1000 rails && \ + useradd rails --uid 1000 --gid 1000 --create-home --shell /bin/bash && \ + chown -R rails:rails db log storage tmp +USER 1000:1000 + +# Entrypoint prepares the database. +ENTRYPOINT ["/rails/bin/docker-entrypoint"] + +# Start server via Thruster by default, this can be overwritten at runtime +EXPOSE 80 +CMD ["./bin/thrust", "./bin/rails", "server"] +``` + +{{< /tab >}} +{{< tab name="Using DOIs" >}} + +```dockerfile {title=Dockerfile} +# syntax=docker/dockerfile:1 +# check=error=true + +# This Dockerfile is designed for production, not development. +# docker build -t app . +# docker run -d -p 80:80 -e RAILS_MASTER_KEY= --name app app + +# For a containerized dev environment, see Dev Containers: https://guides.rubyonrails.org/getting_started_with_devcontainer.html + +# Make sure RUBY_VERSION matches the Ruby version in .ruby-version +ARG RUBY_VERSION=3.4.8 +FROM docker.io/library/ruby:$RUBY_VERSION-slim AS base + +# Rails app lives here +WORKDIR /rails + +# Install base packages +# Replace libpq-dev with sqlite3 if using SQLite, or libmysqlclient-dev if using MySQL +RUN apt-get update -qq && \ + apt-get install --no-install-recommends -y curl libjemalloc2 libvips libpq-dev && \ + rm -rf /var/lib/apt/lists /var/cache/apt/archives + +# Set production environment +ENV RAILS_ENV="production" \ + BUNDLE_DEPLOYMENT="1" \ + BUNDLE_PATH="/usr/local/bundle" \ + BUNDLE_WITHOUT="development" + +# Throw-away build stage to reduce size of final image +FROM base AS build + +# Install packages needed to build gems +RUN apt-get update -qq && \ + apt-get install --no-install-recommends -y build-essential curl git pkg-config libyaml-dev && \ + rm -rf /var/lib/apt/lists /var/cache/apt/archives + +# Install JavaScript dependencies and Node.js for asset compilation +# +# Uncomment the following lines if you are using NodeJS need to compile assets +# +# ARG NODE_VERSION=18.12.0 +# ARG YARN_VERSION=1.22.19 +# ENV PATH=/usr/local/node/bin:$PATH +# RUN curl -sL https://github.com/nodenv/node-build/archive/master.tar.gz | tar xz -C /tmp/ && \ +# /tmp/node-build-master/bin/node-build "${NODE_VERSION}" /usr/local/node && \ +# npm install -g yarn@$YARN_VERSION && \ +# npm install -g mjml && \ +# rm -rf /tmp/node-build-master + +# Install application gems +COPY Gemfile Gemfile.lock ./ +RUN bundle install && \ + rm -rf ~/.bundle/ "${BUNDLE_PATH}"/ruby/*/cache "${BUNDLE_PATH}"/ruby/*/bundler/gems/*/.git && \ + bundle exec bootsnap precompile --gemfile + +# Install node modules +# +# Uncomment the following lines if you are using NodeJS need to compile assets +# +# COPY package.json yarn.lock ./ +# RUN --mount=type=cache,id=yarn,target=/rails/.cache/yarn YARN_CACHE_FOLDER=/rails/.cache/yarn \ +# yarn install --frozen-lockfile + +# Copy application code +COPY . . + +# Precompile bootsnap code for faster boot times +RUN bundle exec bootsnap precompile app/ lib/ + +# Precompiling assets for production without requiring secret RAILS_MASTER_KEY +RUN SECRET_KEY_BASE_DUMMY=1 ./bin/rails assets:precompile + +# Final stage for app image +FROM base + +# Copy built artifacts: gems, application +COPY --from=build "${BUNDLE_PATH}" "${BUNDLE_PATH}" +COPY --from=build /rails /rails + +# Run and own only the runtime files as a non-root user for security +RUN groupadd --system --gid 1000 rails && \ + useradd rails --uid 1000 --gid 1000 --create-home --shell /bin/bash && \ + chown -R rails:rails db log storage tmp +USER 1000:1000 + +# Entrypoint prepares the database. +ENTRYPOINT ["/rails/bin/docker-entrypoint"] + +# Start server via Thruster by default, this can be overwritten at runtime +EXPOSE 80 +CMD ["./bin/thrust", "./bin/rails", "server"] +``` + +{{< /tab >}} +{{< /tabs >}} + +The Dockerfile above assumes you are using Thruster together with Puma as an application server. In case you are using any other server, you can replace the last three lines with the following: + +```dockerfile +# Start the application server +EXPOSE 3000 +CMD ["./bin/rails", "server"] +``` + +This Dockerfile uses a script at `./bin/docker-entrypoint` as the container's entrypoint. This script prepares the database and runs the application server. Below is an example of such a script. + +```bash {title=docker-entrypoint} +#!/bin/bash -e + +# Enable jemalloc for reduced memory usage and latency. +if [ -z "${LD_PRELOAD+x}" ]; then + LD_PRELOAD=$(find /usr/lib -name libjemalloc.so.2 -print -quit) + export LD_PRELOAD +fi + +# If running the rails server then create or migrate existing database +if [ "${@: -2:1}" == "./bin/rails" ] && [ "${@: -1:1}" == "server" ]; then + ./bin/rails db:prepare +fi + +exec "${@}" +``` + +Besides the two files above you will also need a `.dockerignore` file. This file is used to exclude files and directories from the context of the build. Below is an example of a `.dockerignore` file. + +```text {collapse=true,title=".dockerignore"} +# See https://docs.docker.com/engine/reference/builder/#dockerignore-file for more about ignoring files. + +# Ignore git directory. +/.git/ +/.gitignore + +# Ignore bundler config. +/.bundle + +# Ignore all environment files. +/.env* + +# Ignore all default key files. +/config/master.key +/config/credentials/*.key + +# Ignore all logfiles and tempfiles. +/log/* +/tmp/* +!/log/.keep +!/tmp/.keep + +# Ignore pidfiles, but keep the directory. +/tmp/pids/* +!/tmp/pids/.keep + +# Ignore storage (uploaded files in development and any SQLite databases). +/storage/* +!/storage/.keep +/tmp/storage/* +!/tmp/storage/.keep + +# Ignore assets. +/node_modules/ +/app/assets/builds/* +!/app/assets/builds/.keep +/public/assets + +# Ignore CI service files. +/.github + +# Ignore development files +/.devcontainer + +# Ignore Docker-related files +/.dockerignore +/Dockerfile* +``` + +The last optional file that you may want is the `compose.yaml` file, which is used by Docker Compose to define the services that make up the application. Since SQLite is being used as the database, there is no need to define a separate service for the database. The only service required is the Rails application itself. + +```yaml {title=compose.yaml} +services: + web: + build: . + environment: + - RAILS_MASTER_KEY + ports: + - "3000:80" +``` + +You should now have the following files in your application folder: + +- `.dockerignore` +- `compose.yaml` +- `Dockerfile` +- `bin/docker-entrypoint` + +To learn more about the files, see the following: + +- [Dockerfile](/reference/dockerfile) +- [.dockerignore](/reference/dockerfile#dockerignore-file) +- [compose.yaml](/reference/compose-file/_index.md) +- [docker-entrypoint](/reference/dockerfile/#entrypoint) + +### 2. Run the application + +To run the application, run the following command in a terminal inside the application's directory. + +```console +$ RAILS_MASTER_KEY= docker compose up --build +``` + +Open a browser and view the application at [http://localhost:3000](http://localhost:3000). You should see a simple Ruby on Rails application. + +In the terminal, press `ctrl`+`c` to stop the application. + +### 3. Run the application in the background + +You can run the application detached from the terminal by adding the `-d` +option. Inside the `docker-ruby-on-rails` directory, run the following command +in a terminal. + +```console +$ docker compose up --build -d +``` + +Open a browser and view the application at [http://localhost:3000](http://localhost:3000). + +You should see a simple Ruby on Rails application. + +In the terminal, run the following command to stop the application. + +```console +$ docker compose down +``` + +For more information about Compose commands, see the [Compose CLI +reference](/reference/cli/docker/compose/). + +### Summary + +In this section, you learned how you can containerize and run your Ruby +application using Docker. + +Related information: + +- [Docker Compose overview](/manuals/compose/_index.md) + +### Next steps + +In the next section, you'll take a look at how to set up a CI/CD pipeline using GitHub Actions. + +## Automate your builds with GitHub Actions + +### Prerequisites + +Complete all the previous sections of this guide, starting with [Containerize a Ruby on Rails application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. + +If you didn't create a [GitHub repository](https://github.com/new) for your project yet, it is time to do it. After creating the repository, don't forget to [add a remote](https://docs.github.com/en/get-started/getting-started-with-git/managing-remote-repositories) and ensure you can commit and [push your code](https://docs.github.com/en/get-started/using-git/pushing-commits-to-a-remote-repository#about-git-push) to GitHub. + +1. In your project's GitHub repository, open **Settings**, and go to **Secrets and variables** > **Actions**. + +2. Under the **Variables** tab, create a new **Repository variable** named `DOCKER_USERNAME` and your Docker ID as a value. + +3. Create a new [Personal Access Token (PAT)](/manuals/security/access-tokens.md#create-an-access-token) for Docker Hub. You can name this token `docker-tutorial`. Make sure access permissions include Read and Write. + +4. Add the PAT as a **Repository secret** in your GitHub repository, with the name + `DOCKERHUB_TOKEN`. + +### Overview + +GitHub Actions is a CI/CD (Continuous Integration and Continuous Deployment) automation tool built into GitHub. It allows you to define custom workflows for building, testing, and deploying your code when specific events occur (e.g., pushing code, creating a pull request, etc.). A workflow is a YAML-based automation script that defines a sequence of steps to be executed when triggered. Workflows are stored in the `.github/workflows/` directory of a repository. + +In this section, you'll learn how to set up and use GitHub Actions to build your Docker image as well as push it to Docker Hub. You will complete the following steps: + +1. Define the GitHub Actions workflow. +2. Run the workflow. + +### 1. Define the GitHub Actions workflow + +You can create a GitHub Actions workflow by creating a YAML file in the `.github/workflows/` directory of your repository. To do this use your favorite text editor or the GitHub web interface. The following steps show you how to create a workflow file using the GitHub web interface. + +If you prefer to use the GitHub web interface, follow these steps: + +1. Go to your repository on GitHub and then select the **Actions** tab. + +2. Select **set up a workflow yourself**. + + This takes you to a page for creating a new GitHub Actions workflow file in + your repository. By default, the file is created under `.github/workflows/main.yml`, let's change it name to `build.yml`. + +If you prefer to use your text editor, create a new file named `build.yml` in the `.github/workflows/` directory of your repository. + +Add the following content to the file: + +```yaml +name: Build and push Docker image + +on: + push: + branches: + - main + +jobs: + build_and_push: + runs-on: ubuntu-latest + steps: + - name: Login to Docker Hub + uses: docker/login-action@{{% param "login_action_version" %}} + with: + username: ${{ vars.DOCKER_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} + + - name: Build and push + uses: docker/build-push-action@{{% param "build_push_action_version" %}} + with: + push: true + tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest +``` + +Each GitHub Actions workflow includes one or several jobs. Each job consists of steps. Each step can either run a set of commands or use already [existing actions](https://github.com/marketplace?type=actions). The action above has three steps: + +1. [Login to Docker Hub](https://github.com/docker/login-action): Action logs in to Docker Hub using the Docker ID and Personal Access Token (PAT) you created earlier. + +2. [Set up Docker Buildx](https://github.com/docker/setup-buildx-action): Action sets up Docker [Buildx](https://github.com/docker/buildx), a CLI plugin that extends the capabilities of the Docker CLI. + +3. [Build and push](https://github.com/docker/build-push-action): Action builds and pushes the Docker image to Docker Hub. The `tags` parameter specifies the image name and tag. The `latest` tag is used in this example. + +### 2. Run the workflow + +Commit the changes and push them to the `main` branch. This workflow is runs every time you push changes to the `main` branch. You can find more information about workflow triggers [in the GitHub documentation](https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows). + +Go to the **Actions** tab of you GitHub repository. It displays the workflow. Selecting the workflow shows you the breakdown of all the steps. + +When the workflow is complete, go to your [repositories on Docker Hub](https://hub.docker.com/repositories). If you see the new repository in that list, it means the GitHub Actions workflow successfully pushed the image to Docker Hub. + +### Summary + +In this section, you learned how to set up a GitHub Actions workflow for your Ruby on Rails application. + +Related information: + +- [Introduction to GitHub Actions](/guides/gha.md) +- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) +- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) + +### Next steps + +In the next section, you'll learn how you can develop your application using containers. + +## Use containers for Ruby on Rails development + +### Prerequisites + +Complete [Containerize a Ruby on Rails application](containerize.md). + +### Overview + +In this section, you'll learn how to set up a development environment for your containerized application. This includes: + +- Adding a local database and persisting data +- Configuring Compose to automatically update your running Compose services as you edit and save your code + +### Add a local database and persist data + +You can use containers to set up local services, like a database. In this section, you'll update the `compose.yaml` file to define a database service and a volume to persist data. + +In the cloned repository's directory, open the `compose.yaml` file in an IDE or text editor. You need to add the database password file as an environment variable to the server service and specify the secret file to use. + +The following is the updated `compose.yaml` file. + +```yaml {hl_lines="07-25"} +services: + web: + build: . + command: bundle exec rails s -b '0.0.0.0' + ports: + - "3000:3000" + depends_on: + - db + environment: + - RAILS_ENV=test + env_file: "webapp.env" + db: + image: postgres:18 + secrets: + - db-password + environment: + - POSTGRES_PASSWORD_FILE=/run/secrets/db-password + volumes: + - postgres_data:/var/lib/postgresql + +volumes: + postgres_data: +secrets: + db-password: + file: db/password.txt +``` + +> [!NOTE] +> +> To learn more about the instructions in the Compose file, see [Compose file +> reference](/reference/compose-file/). + +Before you run the application using Compose, notice that this Compose file specifies a `password.txt` file to hold the database's password. You must create this file as it's not included in the source repository. + +In the cloned repository's directory, create a new directory named `db` and inside that directory create a file named `password.txt` that contains the password for the database. Using your favorite IDE or text editor, add the following contents to the `password.txt` file. + +```text +mysecretpassword +``` + +Save and close the `password.txt` file. In addition, in the file `webapp.env` you can change the password to connect to the database. + +You should now have the following contents in your `docker-ruby-on-rails` +directory. + +```text +. +├── Dockerfile +├── Gemfile +├── Gemfile.lock +├── README.md +├── Rakefile +├── app/ +├── bin/ +├── compose.yaml +├── config/ +├── config.ru +├── db/ +│ ├── development.sqlite3 +│ ├── migrate +│ ├── password.txt +│ ├── schema.rb +│ └── seeds.rb +├── lib/ +├── log/ +├── public/ +├── storage/ +├── test/ +├── tmp/ +└── vendor +``` + +Now, run the following `docker compose up` command to start your application. + +```console +$ docker compose up --build +``` + +In Ruby on Rails, `db:migrate` is a Rake task that is used to run migrations on the database. Migrations are a way to alter the structure of your database schema over time in a consistent and easy way. + +```console +$ docker exec -it docker-ruby-on-rails-web-1 rake db:migrate RAILS_ENV=test +``` + +You will see a similar message like this: + +`console +== 20240710193146 CreateWhales: migrating ===================================== +-- create_table(:whales) + -> 0.0126s +== 20240710193146 CreateWhales: migrated (0.0127s) ============================ +` + +Refresh in your browser and add the whales. + +Press `ctrl+c` in the terminal to stop your application and run `docker compose up` again, the whales are being persisted. + +### Automatically update services + +Use Compose Watch to automatically update your running Compose services as you +edit and save your code. For more details about Compose Watch, see [Use Compose +Watch](/manuals/compose/how-tos/file-watch.md). + +Open your `compose.yaml` file in an IDE or text editor and then add the Compose +Watch instructions. The following is the updated `compose.yaml` file. + +```yaml {hl_lines="13-16"} +services: + web: + build: . + command: bundle exec rails s -b '0.0.0.0' + ports: + - "3000:3000" + depends_on: + - db + environment: + - RAILS_ENV=test + env_file: "webapp.env" + + develop: + watch: + - action: rebuild + path: . + db: + image: postgres:18 + secrets: + - db-password + environment: + - POSTGRES_PASSWORD_FILE=/run/secrets/db-password + volumes: + - postgres_data:/var/lib/postgresql + +volumes: + postgres_data: +secrets: + db-password: + file: db/password.txt +``` + +Run the following command to run your application with Compose Watch. + +```console +$ docker compose watch +``` + +Any changes to the application's source files on your local machine will now be immediately reflected in the running container. + +Open `docker-ruby-on-rails/app/views/whales/index.html.erb` in an IDE or text editor and update the `Whales` string by adding an exclamation mark. + +```diff +-

    Whales

    ++

    Whales!

    +``` + +Save the changes to `index.html.erb` and then wait a few seconds for the application to rebuild. Go to the application again and verify that the updated text appears. + +Press `ctrl+c` in the terminal to stop your application. + +### Summary + +In this section, you took a look at setting up your Compose file to add a local +database and persist data. You also learned how to use Compose Watch to automatically rebuild and run your container when you update your code. + +Related information: + +- [Compose file reference](/reference/compose-file/) +- [Compose file watch](/manuals/compose/how-tos/file-watch.md) +- [Multi-stage builds](/manuals/build/building/multi-stage.md) + +### Next steps + +In the next section, you'll learn how you can locally test and debug your workloads on Kubernetes before deploying. + +## Test your Ruby on Rails deployment + +### Prerequisites + +- Complete all the previous sections of this guide, starting with [Containerize a Ruby on Rails application](containerize.md). +- [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. + +### Overview + +In this section, you'll learn how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. This lets you to test and debug your workloads on Kubernetes locally before deploying. + +### Create a Kubernetes YAML file + +In your `docker-ruby-on-rails` directory, create a file named +`docker-ruby-on-rails-kubernetes.yaml`. Open the file in an IDE or text editor and add +the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker +username and the name of the repository that you created in [Configure CI/CD for +your Ruby on Rails application](configure-github-actions.md). + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: docker-ruby-on-rails-demo + namespace: default +spec: + replicas: 1 + selector: + matchLabels: + service: ruby-on-rails + template: + metadata: + labels: + service: ruby-on-rails + spec: + containers: + - name: ruby-on-rails-container + image: DOCKER_USERNAME/REPO_NAME + imagePullPolicy: Always +--- +apiVersion: v1 +kind: Service +metadata: + name: docker-ruby-on-rails-demo + namespace: default +spec: + type: NodePort + selector: + service: ruby-on-rails + ports: + - port: 3000 + targetPort: 3000 + nodePort: 30001 +``` + +In this Kubernetes YAML file, there are two objects, separated by the `---`: + +- A Deployment, describing a scalable group of identical pods. In this case, + you'll get just one replica, or copy of your pod. That pod, which is + described under `template`, has just one container in it. The + container is created from the image built by GitHub Actions in [Configure CI/CD for + your Ruby on Rails application](configure-github-actions.md). +- A NodePort service, which will route traffic from port 30001 on your host to + port 8001 inside the pods it routes to, allowing you to reach your app + from the network. + +To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). + +### Deploy and check your application + +1. In a terminal, navigate to `docker-ruby-on-rails` and deploy your application to + Kubernetes. + + ```console + $ kubectl apply -f docker-ruby-on-rails-kubernetes.yaml + ``` + + You should see output that looks like the following, indicating your Kubernetes objects were created successfully. + + ```shell + deployment.apps/docker-ruby-on-rails-demo created + service/docker-ruby-on-rails-demo created + ``` + +2. Make sure everything worked by listing your deployments. + + ```console + $ kubectl get deployments + ``` + + Your deployment should be listed as follows: + + ```shell + NAME READY UP-TO-DATE AVAILABLE AGE + docker-ruby-on-rails-demo 1/1 1 1 15s + ``` + + This indicates all one of the pods you asked for in your YAML are up and running. Do the same check for your services. + + ```console + $ kubectl get services + ``` + + You should get output like the following. + + ```shell + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + kubernetes ClusterIP 10.96.0.1 443/TCP 23h + docker-ruby-on-rails-demo NodePort 10.99.128.230 3000:30001/TCP 75s + ``` + + In addition to the default `kubernetes` service, you can see your `docker-ruby-on-rails-demo` service, accepting traffic on port 30001/TCP. + +3. To create and migrate the database in a Ruby on Rails application running on Kubernetes, you need to follow these steps. + + **Get the Current Pods**: + First, you need to identify the pods running in your Kubernetes cluster. Execute the following command to list the current pods in the `default` namespace: + + ```sh + # Get the current pods in the cluster in the namespace default + $ kubectl get pods + ``` + + This command will display a list of all pods in the `default` namespace. Look for the pod with the prefix `docker-ruby-on-rails-demo-`. Here is an example output: + + ```console + NAME READY STATUS RESTARTS AGE + docker-ruby-on-rails-demo-7cbddb5d6f-qh44l 1/1 Running 2 (22h ago) 9d + ``` + + **Execute the Migration Command**: + Once you've identified the correct pod, use the `kubectl exec` command to run the database migration inside the pod. + + ```sh + $ kubectl exec -it docker-ruby-on-rails-demo-7cbddb5d6f-qh44l -- rails db:migrate RAILS_ENV=development + ``` + + This command opens an interactive terminal session (`-it`) in the specified pod and runs the `rails db:migrate` command with the environment set to development (`RAILS_ENV=development`). + + By following these steps, you ensure that your database is properly migrated within the Ruby on Rails application running in your Kubernetes cluster. This process helps maintain the integrity and consistency of your application's data structure during deployment and updates. + +4. Open the browser and go to [http://localhost:30001](http://localhost:30001), you should see the ruby on rails application working. + +5. Run the following command to tear down your application. + + ```console + $ kubectl delete -f docker-ruby-on-rails-kubernetes.yaml + ``` + +### Summary + +In this section, you learned how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. + +Related information: + +- [Kubernetes documentation](https://kubernetes.io/docs/home/) +- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) +- [Swarm mode overview](/manuals/engine/swarm/_index.md) diff --git a/content/guides/ruby/configure-github-actions.md b/content/guides/ruby/configure-github-actions.md deleted file mode 100644 index 0485bb00eeb4..000000000000 --- a/content/guides/ruby/configure-github-actions.md +++ /dev/null @@ -1,111 +0,0 @@ ---- -title: Automate your builds with GitHub Actions -linkTitle: GitHub Actions CI -weight: 20 -keywords: ci/cd, github actions, ruby, flask -description: Learn how to configure CI/CD using GitHub Actions for your Ruby on Rails application. -aliases: - - /language/ruby/configure-ci-cd/ - - /guides/language/ruby/configure-ci-cd/ - - /guides/ruby/configure-ci-cd/ ---- - -## Prerequisites - -Complete all the previous sections of this guide, starting with [Containerize a Ruby on Rails application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. - -If you didn't create a [GitHub repository](https://github.com/new) for your project yet, it is time to do it. After creating the repository, don't forget to [add a remote](https://docs.github.com/en/get-started/getting-started-with-git/managing-remote-repositories) and ensure you can commit and [push your code](https://docs.github.com/en/get-started/using-git/pushing-commits-to-a-remote-repository#about-git-push) to GitHub. - -1. In your project's GitHub repository, open **Settings**, and go to **Secrets and variables** > **Actions**. - -2. Under the **Variables** tab, create a new **Repository variable** named `DOCKER_USERNAME` and your Docker ID as a value. - -3. Create a new [Personal Access Token (PAT)](/manuals/security/access-tokens.md#create-an-access-token) for Docker Hub. You can name this token `docker-tutorial`. Make sure access permissions include Read and Write. - -4. Add the PAT as a **Repository secret** in your GitHub repository, with the name - `DOCKERHUB_TOKEN`. - -## Overview - -GitHub Actions is a CI/CD (Continuous Integration and Continuous Deployment) automation tool built into GitHub. It allows you to define custom workflows for building, testing, and deploying your code when specific events occur (e.g., pushing code, creating a pull request, etc.). A workflow is a YAML-based automation script that defines a sequence of steps to be executed when triggered. Workflows are stored in the `.github/workflows/` directory of a repository. - -In this section, you'll learn how to set up and use GitHub Actions to build your Docker image as well as push it to Docker Hub. You will complete the following steps: - -1. Define the GitHub Actions workflow. -2. Run the workflow. - -## 1. Define the GitHub Actions workflow - -You can create a GitHub Actions workflow by creating a YAML file in the `.github/workflows/` directory of your repository. To do this use your favorite text editor or the GitHub web interface. The following steps show you how to create a workflow file using the GitHub web interface. - -If you prefer to use the GitHub web interface, follow these steps: - -1. Go to your repository on GitHub and then select the **Actions** tab. - -2. Select **set up a workflow yourself**. - - This takes you to a page for creating a new GitHub Actions workflow file in - your repository. By default, the file is created under `.github/workflows/main.yml`, let's change it name to `build.yml`. - -If you prefer to use your text editor, create a new file named `build.yml` in the `.github/workflows/` directory of your repository. - -Add the following content to the file: - -```yaml -name: Build and push Docker image - -on: - push: - branches: - - main - -jobs: - build_and_push: - runs-on: ubuntu-latest - steps: - - name: Login to Docker Hub - uses: docker/login-action@{{% param "login_action_version" %}} - with: - username: ${{ vars.DOCKER_USERNAME }} - password: ${{ secrets.DOCKERHUB_TOKEN }} - - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} - - - name: Build and push - uses: docker/build-push-action@{{% param "build_push_action_version" %}} - with: - push: true - tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest -``` - -Each GitHub Actions workflow includes one or several jobs. Each job consists of steps. Each step can either run a set of commands or use already [existing actions](https://github.com/marketplace?type=actions). The action above has three steps: - -1. [Login to Docker Hub](https://github.com/docker/login-action): Action logs in to Docker Hub using the Docker ID and Personal Access Token (PAT) you created earlier. - -2. [Set up Docker Buildx](https://github.com/docker/setup-buildx-action): Action sets up Docker [Buildx](https://github.com/docker/buildx), a CLI plugin that extends the capabilities of the Docker CLI. - -3. [Build and push](https://github.com/docker/build-push-action): Action builds and pushes the Docker image to Docker Hub. The `tags` parameter specifies the image name and tag. The `latest` tag is used in this example. - -## 2. Run the workflow - -Commit the changes and push them to the `main` branch. This workflow is runs every time you push changes to the `main` branch. You can find more information about workflow triggers [in the GitHub documentation](https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows). - -Go to the **Actions** tab of you GitHub repository. It displays the workflow. Selecting the workflow shows you the breakdown of all the steps. - -When the workflow is complete, go to your [repositories on Docker Hub](https://hub.docker.com/repositories). If you see the new repository in that list, it means the GitHub Actions workflow successfully pushed the image to Docker Hub. - -## Summary - -In this section, you learned how to set up a GitHub Actions workflow for your Ruby on Rails application. - -Related information: - -- [Introduction to GitHub Actions](/guides/gha.md) -- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) -- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) - -## Next steps - -In the next section, you'll learn how you can develop your application using containers. - diff --git a/content/guides/ruby/containerize.md b/content/guides/ruby/containerize.md deleted file mode 100644 index 5243efd211f7..000000000000 --- a/content/guides/ruby/containerize.md +++ /dev/null @@ -1,392 +0,0 @@ ---- -title: Containerize a Ruby on Rails application -linkTitle: Containerize your app -weight: 10 -keywords: ruby, flask, containerize, initialize -description: Learn how to containerize a Ruby on Rails application. -aliases: - - /language/ruby/build-images/ - - /language/ruby/run-containers/ - - /language/ruby/containerize/ - - /guides/language/ruby/containerize/ ---- - -## Prerequisites - -- You have installed the latest version of [Docker Desktop](/get-started/get-docker.md). -- You have a [Git client](https://git-scm.com/downloads). The examples in this section show the Git CLI, but you can use any client. - -## Overview - -This section walks you through containerizing and running a [Ruby on Rails](https://rubyonrails.org/) application. - -Starting from Rails 7.1 [Docker is supported out of the box](https://guides.rubyonrails.org/7_1_release_notes.html#generate-dockerfiles-for-new-rails-applications). This means that you will get a `Dockerfile`, `.dockerignore` and `bin/docker-entrypoint` files generated for you when you create a new Rails application. - -If you have an existing Rails application, you will need to create the Docker assets manually from the examples below. - -## 1. Create Docker assets - -> [!TIP] -> -> [Gordon](/ai/gordon/), Docker's AI assistant, can generate Docker assets for your project. Ask Gordon to create a Dockerfile, Compose file, and `.dockerignore` tailored to your application. - -Rails 7.1 and newer generates multistage Dockerfile out of the box. Following are two versions of such a file: one using Docker Hardened Images (DHIs) and another using the Docker Official Image (DOIs). Although the Dockerfile is generated automatically, understanding its purpose and functionality is important. Reviewing the following example is highly recommended. - -[Docker Hardened Images (DHIs)](https://docs.docker.com/dhi/) are minimal, secure, and production-ready container base and application images maintained by Docker. DHIs are recommended whenever it is possible for better security. They are designed to reduce vulnerabilities and simplify compliance, freely available to everyone with no subscription required, no usage restrictions, and no vendor lock-in. - -Multistage Dockerfiles help create smaller, more efficient images by separating build and runtime dependencies, ensuring only necessary components are included in the final image. Read more in the [Multi-stage builds guide](/get-started/docker-concepts/building-images/multi-stage-builds/). - - - -{{< tabs >}} -{{< tab name="Using DHIs" >}} - -You must authenticate to `dhi.io` before you can pull Docker Hardened Images. Run `docker login dhi.io` to authenticate. - -```dockerfile {title=Dockerfile} -# syntax=docker/dockerfile:1 -# check=error=true - -# This Dockerfile is designed for production, not development. -# docker build -t app . -# docker run -d -p 80:80 -e RAILS_MASTER_KEY= --name app app - -# For a containerized dev environment, see Dev Containers: https://guides.rubyonrails.org/getting_started_with_devcontainer.html - -# Make sure RUBY_VERSION matches the Ruby version in .ruby-version -ARG RUBY_VERSION=3.4.8 -FROM dhi.io/ruby:$RUBY_VERSION-dev AS base - -# Rails app lives here -WORKDIR /rails - -# Install base packages -# Replace libpq-dev with sqlite3 if using SQLite, or libmysqlclient-dev if using MySQL -RUN apt-get update -qq && \ - apt-get install --no-install-recommends -y curl libjemalloc2 libvips libpq-dev && \ - rm -rf /var/lib/apt/lists /var/cache/apt/archives - -# Set production environment -ENV RAILS_ENV="production" \ - BUNDLE_DEPLOYMENT="1" \ - BUNDLE_PATH="/usr/local/bundle" \ - BUNDLE_WITHOUT="development" - -# Throw-away build stage to reduce size of final image -FROM base AS build - -# Install packages needed to build gems -RUN apt-get update -qq && \ - apt-get install --no-install-recommends -y build-essential curl git pkg-config libyaml-dev && \ - rm -rf /var/lib/apt/lists /var/cache/apt/archives - -# Install JavaScript dependencies and Node.js for asset compilation -# -# Uncomment the following lines if you are using NodeJS need to compile assets -# -# ARG NODE_VERSION=18.12.0 -# ARG YARN_VERSION=1.22.19 -# ENV PATH=/usr/local/node/bin:$PATH -# RUN curl -sL https://github.com/nodenv/node-build/archive/master.tar.gz | tar xz -C /tmp/ && \ -# /tmp/node-build-master/bin/node-build "${NODE_VERSION}" /usr/local/node && \ -# npm install -g yarn@$YARN_VERSION && \ -# npm install -g mjml && \ -# rm -rf /tmp/node-build-master - -# Install application gems -COPY Gemfile Gemfile.lock ./ -RUN bundle install && \ - rm -rf ~/.bundle/ "${BUNDLE_PATH}"/ruby/*/cache "${BUNDLE_PATH}"/ruby/*/bundler/gems/*/.git && \ - bundle exec bootsnap precompile --gemfile - -# Install node modules -# -# Uncomment the following lines if you are using NodeJS need to compile assets -# -# COPY package.json yarn.lock ./ -# RUN --mount=type=cache,id=yarn,target=/rails/.cache/yarn YARN_CACHE_FOLDER=/rails/.cache/yarn \ -# yarn install --frozen-lockfile - -# Copy application code -COPY . . - -# Precompile bootsnap code for faster boot times -RUN bundle exec bootsnap precompile app/ lib/ - -# Precompiling assets for production without requiring secret RAILS_MASTER_KEY -RUN SECRET_KEY_BASE_DUMMY=1 ./bin/rails assets:precompile - -# Final stage for app image -FROM base - -# Copy built artifacts: gems, application -COPY --from=build "${BUNDLE_PATH}" "${BUNDLE_PATH}" -COPY --from=build /rails /rails - -# Run and own only the runtime files as a non-root user for security -RUN groupadd --system --gid 1000 rails && \ - useradd rails --uid 1000 --gid 1000 --create-home --shell /bin/bash && \ - chown -R rails:rails db log storage tmp -USER 1000:1000 - -# Entrypoint prepares the database. -ENTRYPOINT ["/rails/bin/docker-entrypoint"] - -# Start server via Thruster by default, this can be overwritten at runtime -EXPOSE 80 -CMD ["./bin/thrust", "./bin/rails", "server"] -``` - -{{< /tab >}} -{{< tab name="Using DOIs" >}} - -```dockerfile {title=Dockerfile} -# syntax=docker/dockerfile:1 -# check=error=true - -# This Dockerfile is designed for production, not development. -# docker build -t app . -# docker run -d -p 80:80 -e RAILS_MASTER_KEY= --name app app - -# For a containerized dev environment, see Dev Containers: https://guides.rubyonrails.org/getting_started_with_devcontainer.html - -# Make sure RUBY_VERSION matches the Ruby version in .ruby-version -ARG RUBY_VERSION=3.4.8 -FROM docker.io/library/ruby:$RUBY_VERSION-slim AS base - -# Rails app lives here -WORKDIR /rails - -# Install base packages -# Replace libpq-dev with sqlite3 if using SQLite, or libmysqlclient-dev if using MySQL -RUN apt-get update -qq && \ - apt-get install --no-install-recommends -y curl libjemalloc2 libvips libpq-dev && \ - rm -rf /var/lib/apt/lists /var/cache/apt/archives - -# Set production environment -ENV RAILS_ENV="production" \ - BUNDLE_DEPLOYMENT="1" \ - BUNDLE_PATH="/usr/local/bundle" \ - BUNDLE_WITHOUT="development" - -# Throw-away build stage to reduce size of final image -FROM base AS build - -# Install packages needed to build gems -RUN apt-get update -qq && \ - apt-get install --no-install-recommends -y build-essential curl git pkg-config libyaml-dev && \ - rm -rf /var/lib/apt/lists /var/cache/apt/archives - -# Install JavaScript dependencies and Node.js for asset compilation -# -# Uncomment the following lines if you are using NodeJS need to compile assets -# -# ARG NODE_VERSION=18.12.0 -# ARG YARN_VERSION=1.22.19 -# ENV PATH=/usr/local/node/bin:$PATH -# RUN curl -sL https://github.com/nodenv/node-build/archive/master.tar.gz | tar xz -C /tmp/ && \ -# /tmp/node-build-master/bin/node-build "${NODE_VERSION}" /usr/local/node && \ -# npm install -g yarn@$YARN_VERSION && \ -# npm install -g mjml && \ -# rm -rf /tmp/node-build-master - -# Install application gems -COPY Gemfile Gemfile.lock ./ -RUN bundle install && \ - rm -rf ~/.bundle/ "${BUNDLE_PATH}"/ruby/*/cache "${BUNDLE_PATH}"/ruby/*/bundler/gems/*/.git && \ - bundle exec bootsnap precompile --gemfile - -# Install node modules -# -# Uncomment the following lines if you are using NodeJS need to compile assets -# -# COPY package.json yarn.lock ./ -# RUN --mount=type=cache,id=yarn,target=/rails/.cache/yarn YARN_CACHE_FOLDER=/rails/.cache/yarn \ -# yarn install --frozen-lockfile - -# Copy application code -COPY . . - -# Precompile bootsnap code for faster boot times -RUN bundle exec bootsnap precompile app/ lib/ - -# Precompiling assets for production without requiring secret RAILS_MASTER_KEY -RUN SECRET_KEY_BASE_DUMMY=1 ./bin/rails assets:precompile - -# Final stage for app image -FROM base - -# Copy built artifacts: gems, application -COPY --from=build "${BUNDLE_PATH}" "${BUNDLE_PATH}" -COPY --from=build /rails /rails - -# Run and own only the runtime files as a non-root user for security -RUN groupadd --system --gid 1000 rails && \ - useradd rails --uid 1000 --gid 1000 --create-home --shell /bin/bash && \ - chown -R rails:rails db log storage tmp -USER 1000:1000 - -# Entrypoint prepares the database. -ENTRYPOINT ["/rails/bin/docker-entrypoint"] - -# Start server via Thruster by default, this can be overwritten at runtime -EXPOSE 80 -CMD ["./bin/thrust", "./bin/rails", "server"] -``` - -{{< /tab >}} -{{< /tabs >}} - -The Dockerfile above assumes you are using Thruster together with Puma as an application server. In case you are using any other server, you can replace the last three lines with the following: - -```dockerfile -# Start the application server -EXPOSE 3000 -CMD ["./bin/rails", "server"] -``` - -This Dockerfile uses a script at `./bin/docker-entrypoint` as the container's entrypoint. This script prepares the database and runs the application server. Below is an example of such a script. - -```bash {title=docker-entrypoint} -#!/bin/bash -e - -# Enable jemalloc for reduced memory usage and latency. -if [ -z "${LD_PRELOAD+x}" ]; then - LD_PRELOAD=$(find /usr/lib -name libjemalloc.so.2 -print -quit) - export LD_PRELOAD -fi - -# If running the rails server then create or migrate existing database -if [ "${@: -2:1}" == "./bin/rails" ] && [ "${@: -1:1}" == "server" ]; then - ./bin/rails db:prepare -fi - -exec "${@}" -``` - -Besides the two files above you will also need a `.dockerignore` file. This file is used to exclude files and directories from the context of the build. Below is an example of a `.dockerignore` file. - -```text {collapse=true,title=".dockerignore"} -# See https://docs.docker.com/engine/reference/builder/#dockerignore-file for more about ignoring files. - -# Ignore git directory. -/.git/ -/.gitignore - -# Ignore bundler config. -/.bundle - -# Ignore all environment files. -/.env* - -# Ignore all default key files. -/config/master.key -/config/credentials/*.key - -# Ignore all logfiles and tempfiles. -/log/* -/tmp/* -!/log/.keep -!/tmp/.keep - -# Ignore pidfiles, but keep the directory. -/tmp/pids/* -!/tmp/pids/.keep - -# Ignore storage (uploaded files in development and any SQLite databases). -/storage/* -!/storage/.keep -/tmp/storage/* -!/tmp/storage/.keep - -# Ignore assets. -/node_modules/ -/app/assets/builds/* -!/app/assets/builds/.keep -/public/assets - -# Ignore CI service files. -/.github - -# Ignore development files -/.devcontainer - -# Ignore Docker-related files -/.dockerignore -/Dockerfile* -``` - -The last optional file that you may want is the `compose.yaml` file, which is used by Docker Compose to define the services that make up the application. Since SQLite is being used as the database, there is no need to define a separate service for the database. The only service required is the Rails application itself. - -```yaml {title=compose.yaml} -services: - web: - build: . - environment: - - RAILS_MASTER_KEY - ports: - - "3000:80" -``` - -You should now have the following files in your application folder: - -- `.dockerignore` -- `compose.yaml` -- `Dockerfile` -- `bin/docker-entrypoint` - -To learn more about the files, see the following: - -- [Dockerfile](/reference/dockerfile) -- [.dockerignore](/reference/dockerfile#dockerignore-file) -- [compose.yaml](/reference/compose-file/_index.md) -- [docker-entrypoint](/reference/dockerfile/#entrypoint) - -## 2. Run the application - -To run the application, run the following command in a terminal inside the application's directory. - -```console -$ RAILS_MASTER_KEY= docker compose up --build -``` - -Open a browser and view the application at [http://localhost:3000](http://localhost:3000). You should see a simple Ruby on Rails application. - -In the terminal, press `ctrl`+`c` to stop the application. - -## 3. Run the application in the background - -You can run the application detached from the terminal by adding the `-d` -option. Inside the `docker-ruby-on-rails` directory, run the following command -in a terminal. - -```console -$ docker compose up --build -d -``` - -Open a browser and view the application at [http://localhost:3000](http://localhost:3000). - -You should see a simple Ruby on Rails application. - -In the terminal, run the following command to stop the application. - -```console -$ docker compose down -``` - -For more information about Compose commands, see the [Compose CLI -reference](/reference/cli/docker/compose/). - -## Summary - -In this section, you learned how you can containerize and run your Ruby -application using Docker. - -Related information: - -- [Docker Compose overview](/manuals/compose/_index.md) - -## Next steps - -In the next section, you'll take a look at how to set up a CI/CD pipeline using GitHub Actions. - diff --git a/content/guides/ruby/deploy.md b/content/guides/ruby/deploy.md deleted file mode 100644 index 0fe06fb3bb74..000000000000 --- a/content/guides/ruby/deploy.md +++ /dev/null @@ -1,167 +0,0 @@ ---- -title: Test your Ruby on Rails deployment -linkTitle: Test your deployment -weight: 50 -keywords: deploy, kubernetes, ruby -description: Learn how to develop locally using Kubernetes -aliases: - - /language/ruby/deploy/ - - /guides/language/ruby/deploy/ ---- - -## Prerequisites - -- Complete all the previous sections of this guide, starting with [Containerize a Ruby on Rails application](containerize.md). -- [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. - -## Overview - -In this section, you'll learn how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. This lets you to test and debug your workloads on Kubernetes locally before deploying. - -## Create a Kubernetes YAML file - -In your `docker-ruby-on-rails` directory, create a file named -`docker-ruby-on-rails-kubernetes.yaml`. Open the file in an IDE or text editor and add -the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker -username and the name of the repository that you created in [Configure CI/CD for -your Ruby on Rails application](configure-github-actions.md). - -```yaml -apiVersion: apps/v1 -kind: Deployment -metadata: - name: docker-ruby-on-rails-demo - namespace: default -spec: - replicas: 1 - selector: - matchLabels: - service: ruby-on-rails - template: - metadata: - labels: - service: ruby-on-rails - spec: - containers: - - name: ruby-on-rails-container - image: DOCKER_USERNAME/REPO_NAME - imagePullPolicy: Always ---- -apiVersion: v1 -kind: Service -metadata: - name: docker-ruby-on-rails-demo - namespace: default -spec: - type: NodePort - selector: - service: ruby-on-rails - ports: - - port: 3000 - targetPort: 3000 - nodePort: 30001 -``` - -In this Kubernetes YAML file, there are two objects, separated by the `---`: - -- A Deployment, describing a scalable group of identical pods. In this case, - you'll get just one replica, or copy of your pod. That pod, which is - described under `template`, has just one container in it. The - container is created from the image built by GitHub Actions in [Configure CI/CD for - your Ruby on Rails application](configure-github-actions.md). -- A NodePort service, which will route traffic from port 30001 on your host to - port 8001 inside the pods it routes to, allowing you to reach your app - from the network. - -To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). - -## Deploy and check your application - -1. In a terminal, navigate to `docker-ruby-on-rails` and deploy your application to - Kubernetes. - - ```console - $ kubectl apply -f docker-ruby-on-rails-kubernetes.yaml - ``` - - You should see output that looks like the following, indicating your Kubernetes objects were created successfully. - - ```shell - deployment.apps/docker-ruby-on-rails-demo created - service/docker-ruby-on-rails-demo created - ``` - -2. Make sure everything worked by listing your deployments. - - ```console - $ kubectl get deployments - ``` - - Your deployment should be listed as follows: - - ```shell - NAME READY UP-TO-DATE AVAILABLE AGE - docker-ruby-on-rails-demo 1/1 1 1 15s - ``` - - This indicates all one of the pods you asked for in your YAML are up and running. Do the same check for your services. - - ```console - $ kubectl get services - ``` - - You should get output like the following. - - ```shell - NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE - kubernetes ClusterIP 10.96.0.1 443/TCP 23h - docker-ruby-on-rails-demo NodePort 10.99.128.230 3000:30001/TCP 75s - ``` - - In addition to the default `kubernetes` service, you can see your `docker-ruby-on-rails-demo` service, accepting traffic on port 30001/TCP. - -3. To create and migrate the database in a Ruby on Rails application running on Kubernetes, you need to follow these steps. - - **Get the Current Pods**: - First, you need to identify the pods running in your Kubernetes cluster. Execute the following command to list the current pods in the `default` namespace: - - ```sh - # Get the current pods in the cluster in the namespace default - $ kubectl get pods - ``` - - This command will display a list of all pods in the `default` namespace. Look for the pod with the prefix `docker-ruby-on-rails-demo-`. Here is an example output: - - ```console - NAME READY STATUS RESTARTS AGE - docker-ruby-on-rails-demo-7cbddb5d6f-qh44l 1/1 Running 2 (22h ago) 9d - ``` - - **Execute the Migration Command**: - Once you've identified the correct pod, use the `kubectl exec` command to run the database migration inside the pod. - - ```sh - $ kubectl exec -it docker-ruby-on-rails-demo-7cbddb5d6f-qh44l -- rails db:migrate RAILS_ENV=development - ``` - - This command opens an interactive terminal session (`-it`) in the specified pod and runs the `rails db:migrate` command with the environment set to development (`RAILS_ENV=development`). - - By following these steps, you ensure that your database is properly migrated within the Ruby on Rails application running in your Kubernetes cluster. This process helps maintain the integrity and consistency of your application's data structure during deployment and updates. - -4. Open the browser and go to [http://localhost:30001](http://localhost:30001), you should see the ruby on rails application working. - -5. Run the following command to tear down your application. - - ```console - $ kubectl delete -f docker-ruby-on-rails-kubernetes.yaml - ``` - -## Summary - -In this section, you learned how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. - -Related information: - -- [Kubernetes documentation](https://kubernetes.io/docs/home/) -- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) -- [Swarm mode overview](/manuals/engine/swarm/_index.md) diff --git a/content/guides/ruby/develop.md b/content/guides/ruby/develop.md deleted file mode 100644 index 2f2d738b5495..000000000000 --- a/content/guides/ruby/develop.md +++ /dev/null @@ -1,203 +0,0 @@ ---- -title: Use containers for Ruby on Rails development -linkTitle: Develop your app -weight: 40 -keywords: ruby, local, development -description: Learn how to develop your Ruby on Rails application locally. -aliases: - - /language/ruby/develop/ - - /guides/language/ruby/develop/ ---- - -## Prerequisites - -Complete [Containerize a Ruby on Rails application](containerize.md). - -## Overview - -In this section, you'll learn how to set up a development environment for your containerized application. This includes: - -- Adding a local database and persisting data -- Configuring Compose to automatically update your running Compose services as you edit and save your code - -## Add a local database and persist data - -You can use containers to set up local services, like a database. In this section, you'll update the `compose.yaml` file to define a database service and a volume to persist data. - -In the cloned repository's directory, open the `compose.yaml` file in an IDE or text editor. You need to add the database password file as an environment variable to the server service and specify the secret file to use. - -The following is the updated `compose.yaml` file. - -```yaml {hl_lines="07-25"} -services: - web: - build: . - command: bundle exec rails s -b '0.0.0.0' - ports: - - "3000:3000" - depends_on: - - db - environment: - - RAILS_ENV=test - env_file: "webapp.env" - db: - image: postgres:18 - secrets: - - db-password - environment: - - POSTGRES_PASSWORD_FILE=/run/secrets/db-password - volumes: - - postgres_data:/var/lib/postgresql - -volumes: - postgres_data: -secrets: - db-password: - file: db/password.txt -``` - -> [!NOTE] -> -> To learn more about the instructions in the Compose file, see [Compose file -> reference](/reference/compose-file/). - -Before you run the application using Compose, notice that this Compose file specifies a `password.txt` file to hold the database's password. You must create this file as it's not included in the source repository. - -In the cloned repository's directory, create a new directory named `db` and inside that directory create a file named `password.txt` that contains the password for the database. Using your favorite IDE or text editor, add the following contents to the `password.txt` file. - -```text -mysecretpassword -``` - -Save and close the `password.txt` file. In addition, in the file `webapp.env` you can change the password to connect to the database. - -You should now have the following contents in your `docker-ruby-on-rails` -directory. - -```text -. -├── Dockerfile -├── Gemfile -├── Gemfile.lock -├── README.md -├── Rakefile -├── app/ -├── bin/ -├── compose.yaml -├── config/ -├── config.ru -├── db/ -│ ├── development.sqlite3 -│ ├── migrate -│ ├── password.txt -│ ├── schema.rb -│ └── seeds.rb -├── lib/ -├── log/ -├── public/ -├── storage/ -├── test/ -├── tmp/ -└── vendor -``` - -Now, run the following `docker compose up` command to start your application. - -```console -$ docker compose up --build -``` - -In Ruby on Rails, `db:migrate` is a Rake task that is used to run migrations on the database. Migrations are a way to alter the structure of your database schema over time in a consistent and easy way. - -```console -$ docker exec -it docker-ruby-on-rails-web-1 rake db:migrate RAILS_ENV=test -``` - -You will see a similar message like this: - -`console -== 20240710193146 CreateWhales: migrating ===================================== --- create_table(:whales) - -> 0.0126s -== 20240710193146 CreateWhales: migrated (0.0127s) ============================ -` - -Refresh in your browser and add the whales. - -Press `ctrl+c` in the terminal to stop your application and run `docker compose up` again, the whales are being persisted. - -## Automatically update services - -Use Compose Watch to automatically update your running Compose services as you -edit and save your code. For more details about Compose Watch, see [Use Compose -Watch](/manuals/compose/how-tos/file-watch.md). - -Open your `compose.yaml` file in an IDE or text editor and then add the Compose -Watch instructions. The following is the updated `compose.yaml` file. - -```yaml {hl_lines="13-16"} -services: - web: - build: . - command: bundle exec rails s -b '0.0.0.0' - ports: - - "3000:3000" - depends_on: - - db - environment: - - RAILS_ENV=test - env_file: "webapp.env" - - develop: - watch: - - action: rebuild - path: . - db: - image: postgres:18 - secrets: - - db-password - environment: - - POSTGRES_PASSWORD_FILE=/run/secrets/db-password - volumes: - - postgres_data:/var/lib/postgresql - -volumes: - postgres_data: -secrets: - db-password: - file: db/password.txt -``` - -Run the following command to run your application with Compose Watch. - -```console -$ docker compose watch -``` - -Any changes to the application's source files on your local machine will now be immediately reflected in the running container. - -Open `docker-ruby-on-rails/app/views/whales/index.html.erb` in an IDE or text editor and update the `Whales` string by adding an exclamation mark. - -```diff --

    Whales

    -+

    Whales!

    -``` - -Save the changes to `index.html.erb` and then wait a few seconds for the application to rebuild. Go to the application again and verify that the updated text appears. - -Press `ctrl+c` in the terminal to stop your application. - -## Summary - -In this section, you took a look at setting up your Compose file to add a local -database and persist data. You also learned how to use Compose Watch to automatically rebuild and run your container when you update your code. - -Related information: - -- [Compose file reference](/reference/compose-file/) -- [Compose file watch](/manuals/compose/how-tos/file-watch.md) -- [Multi-stage builds](/manuals/build/building/multi-stage.md) - -## Next steps - -In the next section, you'll learn how you can locally test and debug your workloads on Kubernetes before deploying. diff --git a/content/guides/rust/_index.md b/content/guides/rust/_index.md index 483040adbfce..62291a1ed9ae 100644 --- a/content/guides/rust/_index.md +++ b/content/guides/rust/_index.md @@ -5,17 +5,25 @@ description: Containerize Rust apps using Docker keywords: Docker, getting started, Rust, language summary: | This guide covers how to containerize Rust applications using Docker. -toc_min: 1 -toc_max: 2 aliases: - /language/rust/ - /guides/language/rust/ -languages: [rust] -tags: [dhi] + - /language/rust/build-images/ + - /language/rust/run-containers/ + - /language/rust/develop/ + - /language/rust/configure-ci-cd/ + - /language/rust/deploy/ + - /guides/rust/build-images/ + - /guides/rust/configure-ci-cd/ + - /guides/rust/deploy/ + - /guides/rust/develop/ + - /guides/rust/run-containers/ params: + tags: [cicd] time: 20 minutes --- + The Rust language-specific guide teaches you how to create a containerized Rust application using Docker. In this guide, you'll learn how to: - Containerize a Rust application @@ -29,3 +37,1144 @@ The Rust language-specific guide teaches you how to create a containerized Rust After completing the Rust modules, you should be able to containerize your own Rust application based on the examples and instructions provided in this guide. Start with building your first Rust image. + +## Build your Rust image + +### Prerequisites + +- You have installed the latest version of [Docker Desktop](/get-started/get-docker.md). +- You have a [git client](https://git-scm.com/downloads). The examples in this section use a command-line based git client, but you can use any client. + +### Overview + +This guide walks you through building your first Rust image. An image +includes everything needed to run an application - the code or binary, runtime, +dependencies, and any other file system objects required. + +### Get the sample application + +Clone the sample application to use with this guide. Open a terminal, change directory to a directory that you want to work in, and run the following command to clone the repository: + +```console +$ git clone https://github.com/docker/docker-rust-hello && cd docker-rust-hello +``` + +### Choose a base image + +> [!TIP] +> +> [Gordon](/ai/gordon/), Docker's AI assistant, can generate Docker assets for your project. Ask Gordon to create a Dockerfile, Compose file, and `.dockerignore` tailored to your application. + +Before editing your Dockerfile, you need to choose a base image. You can use the [Rust Docker Official Image](https://hub.docker.com/_/rust), +or a [Docker Hardened Image (DHI)](https://hub.docker.com/hardened-images/catalog/dhi/rust). + +Docker Hardened Images (DHIs) are minimal, secure, and production-ready base images maintained by Docker. +They help reduce vulnerabilities and simplify compliance. For more details, see [Docker Hardened Images](/dhi/). + +{{< tabs >}} +{{< tab name="Using Docker Hardened Images" >}} + +Docker Hardened Images (DHIs) are publicly available and can be used directly as base images. +To pull Docker Hardened Images, authenticate once with Docker: + +```bash +docker login dhi.io +``` + +Use DHIs from the dhi.io registry, for example: + +```bash +FROM dhi.io/rust:${RUST_VERSION}-alpine3.22-dev AS build +``` + +The following Dockerfile uses a Rust DHI as the build base image: + +```dockerfile {title=Dockerfile} +# Make sure RUST_VERSION matches the Rust version +ARG RUST_VERSION=1.92 +ARG APP_NAME=docker-rust-hello + +################################################################################ +# Create a stage for building the application. +################################################################################ + +FROM dhi.io/rust:${RUST_VERSION}-alpine3.22-dev AS build +ARG APP_NAME +WORKDIR /app + +# Install host build dependencies. +RUN apk add --no-cache clang lld musl-dev git + +# Build the application. +RUN --mount=type=bind,source=src,target=src \ + --mount=type=bind,source=Cargo.toml,target=Cargo.toml \ + --mount=type=bind,source=Cargo.lock,target=Cargo.lock \ + --mount=type=cache,target=/app/target/ \ + --mount=type=cache,target=/usr/local/cargo/git/db \ + --mount=type=cache,target=/usr/local/cargo/registry/ \ + cargo build --locked --release && \ + cp ./target/release/$APP_NAME /bin/server + +################################################################################ +# Create a new stage for running the application that contains the minimal +# We use dhi.io/static for the final stage because it’s a minimal Docker Hardened Image runtime (basically “just # enough OS to run the binary”), which helps keep the image small and with a lower attack surface compared to a # # full Alpine/Debian runtime. +################################################################################ + +FROM dhi.io/static:20250419 AS final + +# Copy the executable from the "build" stage. +COPY --from=build /bin/server /bin/ + +# Configure rocket to listen on all interfaces. +ENV ROCKET_ADDRESS=0.0.0.0 + +# Expose the port that the application listens on. +EXPOSE 8000 + +# What the container should run when it is started. +CMD ["/bin/server"] + +``` + +{{< /tab >}} +{{< tab name="Using the Docker Official Images" >}} + +```dockerfile {title=Dockerfile} +# Pin the Rust toolchain version used in the build stage. +ARG RUST_VERSION=1.92 + +# Name of the compiled binary produced by Cargo (must match Cargo.toml package name). +ARG APP_NAME=docker-rust-hello + +################################################################################ +# Build stage (DOI Rust image) +# This stage compiles the application. +################################################################################ + +FROM docker.io/library/rust:${RUST_VERSION}-alpine AS build + +# Re-declare args inside the stage if you want to use them here. +ARG APP_NAME + +# All build steps happen inside /app. +WORKDIR /app + +# Install build dependencies needed to compile Rust crates on Alpine +RUN apk add --no-cache clang lld musl-dev git + +# Build the application +RUN --mount=type=bind,source=src,target=src \ + --mount=type=bind,source=Cargo.toml,target=Cargo.toml \ + --mount=type=bind,source=Cargo.lock,target=Cargo.lock \ + --mount=type=cache,target=/app/target/ \ + --mount=type=cache,target=/usr/local/cargo/git/db \ + --mount=type=cache,target=/usr/local/cargo/registry/ \ + cargo build --locked --release && \ + cp ./target/release/$APP_NAME /bin/server + +################################################################################ +# Runtime stage (DOI Alpine image) +# This stage runs the already-compiled binary with minimal dependencies. +################################################################################ + +FROM docker.io/library/alpine:3.18 AS final + +# Create a non-privileged user (recommended best practice) +ARG UID=10001 +RUN adduser \ + --disabled-password \ + --gecos "" \ + --home "/nonexistent" \ + --shell "/sbin/nologin" \ + --no-create-home \ + --uid "${UID}" \ + appuser + +# Drop privileges for runtime. +USER appuser + +# Copy only the compiled binary from the build stage. +COPY --from=build /bin/server /bin/ + +# Rocket: listen on all interfaces inside the container. +ENV ROCKET_ADDRESS=0.0.0.0 + +# Document the port your app listens on. +EXPOSE 8000 + +# Start the application. +CMD ["/bin/server"] +``` +{{< /tab >}} +{{< /tabs >}} + + + +For building an image, only the Dockerfile is necessary. Open the Dockerfile +in your favorite IDE or text editor and see what it contains. To learn more +about Dockerfiles, see the [Dockerfile reference](/reference/dockerfile.md). + +### .dockerignore file + +The [`.dockerignore`](/reference/dockerfile.md#dockerignore-file) file specifies patterns and paths that you don't want copied into the image in order to keep the image as small as possible. Open up the `.dockerignore` file in your favorite IDE or text editor to review its contents. + +### Build an image + +Now that you’ve created the Dockerfile, you can build the image. To do this, use +the `docker build` command. The `docker build` command builds Docker images from +a Dockerfile and a context. A build's context is the set of files located in +the specified PATH or URL. The Docker build process can access any of the files +located in this context. + +The build command optionally takes a `--tag` flag. The tag sets the name of the +image and an optional tag in the format `name:tag`. If you don't pass a tag, +Docker uses "latest" as its default tag. + +Build the Docker image. + +```console +$ docker build --tag docker-rust-image-dhi . +``` + +You should see output like the following. + +```console +[+] Building 1.4s (13/13) FINISHED docker:desktop-linux + => [internal] load build definition from Dockerfile 0.0s + => => transferring dockerfile: 1.67kB 0.0s + => [internal] load metadata for dhi.io/static:20250419 1.1s + => [internal] load metadata for dhi.io/rust:1.92-alpine3.22-dev 1.2s + => [auth] static:pull token for dhi.io 0.0s + => [auth] rust:pull token for dhi.io 0.0s + => [internal] load .dockerignore 0.0s + => => transferring context: 646B 0.0s + => [build 1/3] FROM dhi.io/rust:1.92-alpine3.22-dev@sha256:49eb72825a9e15fe48f2c4875a63c7e7f52a5b430bb52b8254b91d132aa5bf38 0.0s + => => resolve dhi.io/rust:1.92-alpine3.22-dev@sha256:49eb72825a9e15fe48f2c4875a63c7e7f52a5b430bb52b8254b91d132aa5bf38 0.0s + => [final 1/2] FROM dhi.io/static:20250419@sha256:74fc43fa240887b8159970e434244039aab0c6efaaa9cf044004cdc22aa2a34d 0.0s + => => resolve dhi.io/static:20250419@sha256:74fc43fa240887b8159970e434244039aab0c6efaaa9cf044004cdc22aa2a34d 0.0s + => [internal] load build context 0.0s + => => transferring context: 117B 0.0s + => CACHED [build 2/3] WORKDIR /build 0.0s + => CACHED [build 3/3] RUN --mount=type=bind,source=src,target=src --mount=type=bind,source=Cargo.toml,target=Cargo.toml --mount=type=bind,source=Cargo.lock,target=Cargo 0.0s + => CACHED [final 2/2] COPY --from=build /build/target/release/docker-rust-hello /server 0.0s + => exporting to image 0.1s + => => exporting layers 0.0s + => => exporting manifest sha256:cc937bbdd712ef6e5445501f77e02ef8455ef64c567598786d46b7b21a4d4fa8 0.0s + => => exporting config sha256:077507b483af4b5e1a928e527e4bb3a4aaf0557e1eea81cd39465f564c187669 0.0s + => => exporting attestation manifest sha256:11b60e7608170493da1fdd88c120e2d2957f2a72a22edbc9cfbdd0dd37d21f89 0.0s + => => exporting manifest list sha256:99a1b925a8d6ebf80e376b8a1e50cd806ec42d194479a3375e1cd9d2911b4db9 0.0s + => => naming to docker.io/library/docker-rust-image-dhi:latest 0.0s + => => unpacking to docker.io/library/docker-rust-image-dhi:latest 0.0s + +View build details: docker-desktop://dashboard/build/desktop-linux/desktop-linux/yczk0ijw8kc5g20e8nbc8r6lj +``` + +### View local images + +To see a list of images you have on your local machine, you have two options. One is to use the Docker CLI and the other is to use [Docker Desktop](/manuals/desktop/use-desktop/images.md). As you are working in the terminal already, take a look at listing images using the CLI. + +To list images, run the `docker images` command. + +```console +$ docker images +IMAGE ID DISK USAGE CONTENT SIZE EXTRA +docker-rust-image-dhi:latest 99a1b925a8d6 11.6MB 2.45MB U +``` + +You should see at least one image listed, including the image you just built `docker-rust-image-dhi:latest`. + +### Tag images + +As mentioned earlier, an image name is made up of slash-separated name components. Name components may contain lowercase letters, digits, and separators. A separator can include a period, one or two underscores, or one or more dashes. A name component may not start or end with a separator. + +An image is made up of a manifest and a list of layers. Don't worry too much about manifests and layers at this point other than a "tag" points to a combination of these artifacts. You can have multiple tags for an image. Create a second tag for the image you built and take a look at its layers. + +To create a new tag for the image you built, run the following command. + +```console +$ docker tag docker-rust-image-dhi:latest docker-rust-image-dhi:v1.0.0 +``` + +The `docker tag` command creates a new tag for an image. It doesn't create a new image. The tag points to the same image and is just another way to reference the image. + +Now, run the `docker images` command to see a list of the local images. + +```console +$ docker images +IMAGE ID DISK USAGE CONTENT SIZE EXTRA +docker-rust-image-dhi:latest 99a1b925a8d6 11.6MB 2.45MB U +docker-rust-image-dhi:v1.0.0 99a1b925a8d6 11.6MB 2.45MB U +``` + +You can see that two images start with `docker-rust-image-dhi`. You know they're the same image because if you take a look at the `IMAGE ID` column, you can see that the values are the same for the two images. + +Remove the tag you just created. To do this, use the `rmi` command. The `rmi` command stands for remove image. + +```console +$ docker rmi docker-rust-image-dhi:v1.0.0 +Untagged: docker-rust-image-dhi:v1.0.0 +``` + +Note that the response from Docker tells you that Docker didn't remove the image, but only "untagged" it. You can check this by running the `docker images` command. + +```console +$ docker images +IMAGE ID DISK USAGE CONTENT SIZE EXTRA +docker-rust-image-dhi:latest 99a1b925a8d6 11.6MB 2.45MB U +``` + +Docker removed the image tagged with `:v1.0.0`, but the `docker-rust-image-dhi:latest` tag is available on your machine. + +### Summary + +This section showed how to create a Dockerfile and `.dockerignore` file for a Rust application, build an image, and tag and list images. + +Related information: + +- [Dockerfile reference](/reference/dockerfile.md) +- [.dockerignore file](/reference/dockerfile.md#dockerignore-file) +- [docker build CLI reference](/reference/cli/docker/buildx/build/) +- [Docker Hardened Images](/dhi/) + +### Next steps + +In the next section learn how to run your image as a container. + +## Run your Rust image as a container + +### Prerequisite + +You have completed [Build your Rust image](build-images.md) and you have built an image. + +### Overview + +A container is a normal operating system process except that Docker isolates this process so that it has its own file system, its own networking, and its own isolated process tree separate from the host. + +To run an image inside of a container, you use the `docker run` command. The `docker run` command requires one parameter which is the name of the image. + +### Run an image + +Use `docker run` to run the image you built in [Build your Rust image](build-images.md). + +```console +$ docker run docker-rust-image-dhi +``` + +After running this command, you’ll notice that you weren't returned to the command prompt. This is because your application is a server that runs in a loop waiting for incoming requests without returning control back to the OS until you stop the container. + +Open a new terminal then make a request to the server using the `curl` command. + +```console +$ curl http://localhost:8000 +``` + +You should see output like the following. + +```console +curl: (7) Failed to connect to localhost port 8000 after 2236 ms: Couldn't connect to server +``` + +As you can see, your `curl` command failed. This means you weren't able to connect to the localhost on port 8000. This is normal because your container is running in isolation which includes networking. Stop the container and restart with port 8000 published on your local network. + +To stop the container, press ctrl-c. This will return you to the terminal prompt. + +To publish a port for your container, you’ll use the `--publish` flag (`-p` for short) on the `docker run` command. The format of the `--publish` command is `[host port]:[container port]`. So, if you wanted to expose port 8000 inside the container to port 3001 outside the container, you would pass `3001:8000` to the `--publish` flag. + +You didn't specify a port when running the application in the container and the default is 8000. If you want your previous request going to port 8000 to work, you can map the host's port 3001 to the container's port 8000: + +```console +$ docker run --publish 3001:8000 docker-rust-image-dhi +``` + +Now, rerun the curl command. Remember to open a new terminal. + +```console +$ curl http://localhost:3001 +``` + +You should see output like the following. + +```console +Hello, Docker! +``` + +Success! You were able to connect to the application running inside of your container on port 8000. Switch back to the terminal where your container is running and stop it. + +Press ctrl-c to stop the container. + +### Run in detached mode + +This is great so far, but your sample application is a web server and you don't have to be connected to the container. Docker can run your container in detached mode or in the background. To do this, you can use the `--detach` or `-d` for short. Docker starts your container the same as before but this time will "detach" from the container and return you to the terminal prompt. + +```console +$ docker run -d -p 3001:8000 docker-rust-image-dhi +3e4830e7f01304811d97dd3469d47a0c7a916a8b6c28ce0ef19c6f689a521144 +``` + +Docker started your container in the background and printed the Container ID on the terminal. + +Again, make sure that your container is running properly. Run the curl command again. + +```console +$ curl http://localhost:3001 +``` + +You should see output like the following. + +```console +Hello, Docker! +``` + +### List containers + +Since you ran your container in the background, how do you know if your container is running or what other containers are running on your machine? Well, to see a list of containers running on your machine, run `docker ps`. This is similar to how you use the ps command in Linux to see a list of processes. + +You should see output like the following. + +```console +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +3e4830e7f013 docker-rust-image-dhi "/server" 23 seconds ago Up 22 seconds 0.0.0.0:3001->8000/tcp, [::]:3001->8000/tcp youthful_lamport +``` + +The `docker ps` command provides a bunch of information about your running containers. You can see the container ID, the image running inside the container, the command that was used to start the container, when it was created, the status, ports that were exposed, and the name of the container. + +You are probably wondering where the name of your container is coming from. Since you didn’t provide a name for the container when you started it, Docker generated a random name. You’ll fix this in a minute, but first you need to stop the container. To stop the container, run the `docker stop` command which does just that, stops the container. You need to pass the name of the container or you can use the container ID. + +```console +$ docker stop youthful_lamport +youthful_lamport +``` + +Now, rerun the `docker ps` command to see a list of running containers. + +```console +$ docker ps +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +``` + +### Stop, start, and name containers + +You can start, stop, and restart Docker containers. When you stop a container, it's not removed, but the status is changed to stopped and the process inside the container is stopped. When you ran the `docker ps` command in the previous module, the default output only shows running containers. When you pass the `--all` or `-a` for short, you see all containers on your machine, irrespective of their start or stop status. + +```console +$ docker ps -a +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +3e4830e7f013 docker-rust-image-dhi "/server" About a minute ago Exited (0) 28 seconds ago youthful_lamport +60009b7eaf40 docker-rust-image-dhi "/server" 2 minutes ago Exited (0) About a minute ago sharp_noyce +152e1d7d9eea docker-rust-image-dhi "/server ." 4 minutes ago Exited (0) 2 minutes ago magical_bhabha +``` + +You should now see several containers listed. These are containers that you started and stopped but you haven't removed. + +Restart the container that you just stopped. Locate the name of the container you just stopped and replace the name of the container in following restart command. + +```console +$ docker restart youthful_lamport +``` + +Now list all the containers again using the `docker ps --all` command. + +```console +$ docker ps --all +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +3e4830e7f013 docker-rust-image-dhi "/server" 3 minutes ago Up 7 seconds 0.0.0.0:3001->8000/tcp, [::]:3001->8000/tcp youthful_lamport +60009b7eaf40 docker-rust-image-dhi "/server" 4 minutes ago Exited (0) 3 minutes ago sharp_noyce +152e1d7d9eea docker-rust-image-dhi "/server ." 5 minutes ago Exited (0) 4 minutes ago magical_bhabha +``` + +Notice that the container you just restarted has been started in detached mode. Also, observe the status of the container is "Up X seconds". When you restart a container, it starts with the same flags or commands that it was originally started with. + +Now, stop and remove all of your containers and take a look at fixing the random naming issue. Stop the container you just started. Find the name of your running container and replace the name in the following command with the name of the container on your system. + +```console +$ docker stop youthful_lamport +youthful_lamport +``` + +Now that you have stopped all of your containers, remove them. When you remove a container, it's no longer running, nor is it in the stopped status, but the process inside the container has been stopped and the metadata for the container has been removed. + +To remove a container, run the `docker rm` command with the container name. You can pass multiple container names to the command using a single command. Again, replace the container names in the following command with the container names from your system. + +```console +$ docker rm youthful_lamport friendly_montalcini tender_bose +youthful_lamport +sharp_noyce +magical_bhabha +``` + +Run the `docker ps --all` command again to see that Docker removed all containers. + +Now, it's time to address the random naming issue. Standard practice is to name your containers for the simple reason that it's easier to identify what's running in the container and what application or service it's associated with. + +To name a container, pass the `--name` flag to the `docker run` command. + +```console +$ docker run -d -p 3001:8000 --name docker-rust-container docker-rust-image-dhi +1aa5d46418a68705c81782a58456a4ccdb56a309cb5e6bd399478d01eaa5cdda +$ docker ps +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +219b2e3c7c38 docker-rust-image-dhi "/server" 6 seconds ago Up 5 seconds 0.0.0.0:3001->8000/tcp, [::]:3001->8000/tcp docker-rust-container +``` + +Now you can identify your container based on the name. + +### Summary + +In this section, you took a look at running containers. You also took a look at managing containers by starting, stopping, and restarting them. And finally, you looked at naming your containers so they are more identifiable. + +Related information: + +- [docker run CLI reference](/reference/cli/docker/container/run/) + +### Next steps + +In the next section, you’ll learn how to run a database in a container and connect it to a Rust application. + +## Develop your Rust application + +### Prerequisites + +- You have installed the latest version of [Docker Desktop](/get-started/get-docker.md). +- You have completed the walkthroughs in the Docker Desktop [Learning Center](/manuals/desktop/use-desktop/_index.md) to learn about Docker concepts. +- You have a [git client](https://git-scm.com/downloads). The examples in this section use a command-line based git client, but you can use any client. + +### Overview + +In this section, you’ll learn how to use volumes and networking in Docker. You’ll also use Docker to build your images and Docker Compose to make everything a whole lot easier. + +First, you’ll take a look at running a database in a container and how you can use volumes and networking to persist your data and let your application talk with the database. Then you’ll pull everything together into a Compose file which lets you set up and run a local development environment with one command. + +### Run a database in a container + +Instead of downloading PostgreSQL, installing, configuring, and then running the PostgreSQL database as a service, you can use the Docker Official Image for PostgreSQL and run it in a container. + +Before you run PostgreSQL in a container, create a volume that Docker can manage to store your persistent data and configuration. Use the named volumes feature that Docker provides instead of using bind mounts. + +Run the following command to create your volume. + +```console +$ docker volume create db-data +``` + +Now create a network that your application and database will use to talk to each other. The network is called a user-defined bridge network and gives you a nice DNS lookup service which you can use when creating your connection string. + +```console +$ docker network create postgresnet +``` + +Now you can run PostgreSQL in a container and attach to the volume and network that you created previously. Docker pulls the image from Hub and runs it for you locally. +In the following command, option `--mount` is for starting the container with a volume. For more information, see [Docker volumes](/manuals/engine/storage/volumes.md). + +```console +$ docker run --rm -d --mount \ + "type=volume,src=db-data,target=/var/lib/postgresql" \ + -p 5432:5432 \ + --network postgresnet \ + --name db \ + -e POSTGRES_PASSWORD=mysecretpassword \ + -e POSTGRES_DB=example \ + postgres:18 +``` + +Now, make sure that your PostgreSQL database is running and that you can connect to it. Connect to the running PostgreSQL database inside the container. + +```console +$ docker exec -it db psql -U postgres +``` + +You should see output like the following. + +```console +psql (15.3 (Debian 15.3-1.pgdg110+1)) +Type "help" for help. + +postgres=# +``` + +In the previous command, you logged in to the PostgreSQL database by passing the `psql` command to the `db` container. Press ctrl-d to exit the PostgreSQL interactive terminal. + +### Get and run the sample application + +For the sample application, you'll use a variation of the backend from the react-rust-postgres application from [Awesome Compose](https://github.com/docker/awesome-compose/tree/master/react-rust-postgres). + +1. Clone the sample application repository using the following command. + + ```console + $ git clone https://github.com/docker/docker-rust-postgres + ``` + +2. In the cloned repository's directory, create a `Dockerfile`. This application includes a `migrations` directory (in addition to `src`) to initialize the database, so the Dockerfile includes a bind mount for that directory in the build stage. + + ```dockerfile {hl_lines="28"} + # syntax=docker/dockerfile:1 + + # Comments are provided throughout this file to help you get started. + # If you need more help, visit the Dockerfile reference guide at + # https://docs.docker.com/reference/dockerfile/ + + ################################################################################ + # Create a stage for building the application. + + ARG RUST_VERSION=1.70.0 + ARG APP_NAME=react-rust-postgres + FROM rust:${RUST_VERSION}-slim-bullseye AS build + ARG APP_NAME + WORKDIR /app + + # Build the application. + # Leverage a cache mount to /usr/local/cargo/registry/ + # for downloaded dependencies and a cache mount to /app/target/ for + # compiled dependencies which will speed up subsequent builds. + # Leverage a bind mount to the src directory to avoid having to copy the + # source code into the container. Once built, copy the executable to an + # output directory before the cache mounted /app/target is unmounted. + RUN --mount=type=bind,source=src,target=src \ + --mount=type=bind,source=Cargo.toml,target=Cargo.toml \ + --mount=type=bind,source=Cargo.lock,target=Cargo.lock \ + --mount=type=cache,target=/app/target/ \ + --mount=type=cache,target=/usr/local/cargo/registry/ \ + --mount=type=bind,source=migrations,target=migrations \ + < + **Actions**. + +3. Create a new **Repository variable** named `DOCKER_USERNAME` and your Docker ID as a value. + +4. Create a new [Personal Access Token (PAT)](/manuals/security/access-tokens.md#create-an-access-token) for Docker Hub. You can name this token `docker-tutorial`. Make sure access permissions include Read and Write. + +5. Add the PAT as a **Repository secret** in your GitHub repository, with the name + `DOCKERHUB_TOKEN`. + +6. In your local repository on your machine, run the following command to change + the origin to the repository you just created. Make sure you change + `your-username` to your GitHub username and `your-repository` to the name of + the repository you created. + + ```console + $ git remote set-url origin https://github.com/your-username/your-repository.git + ``` + +7. Run the following commands to stage, commit, and push your local repository to GitHub. + + ```console + $ git add -A + $ git commit -m "my commit" + $ git push -u origin main + ``` + +### Step two: Set up the workflow + +Set up your GitHub Actions workflow for building, testing, and pushing the image +to Docker Hub. + +1. Go to your repository on GitHub and then select the **Actions** tab. + +2. Select **set up a workflow yourself**. + + This takes you to a page for creating a new GitHub actions workflow file in + your repository, under `.github/workflows/main.yml` by default. + +3. In the editor window, copy and paste the following YAML configuration. + + ```yaml + name: ci + + on: + push: + branches: + - main + + jobs: + build: + runs-on: ubuntu-latest + steps: + - name: Login to Docker Hub + uses: docker/login-action@{{% param "login_action_version" %}} + with: + username: ${{ vars.DOCKER_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} + + - name: Build and push + uses: docker/build-push-action@{{% param "build_push_action_version" %}} + with: + push: true + tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest + ``` + + For more information about the YAML syntax for `docker/build-push-action`, + refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). + +### Step three: Run the workflow + +Save the workflow file and run the job. + +1. Select **Commit changes...** and push the changes to the `main` branch. + + After pushing the commit, the workflow starts automatically. + +2. Go to the **Actions** tab. It displays the workflow. + + Selecting the workflow shows you the breakdown of all the steps. + +3. When the workflow is complete, go to your + [repositories on Docker Hub](https://hub.docker.com/repositories). + + If you see the new repository in that list, it means the GitHub Actions + successfully pushed the image to Docker Hub. + +### Summary + +In this section, you learned how to set up a GitHub Actions workflow for your Rust application. + +Related information: + +- [Introduction to GitHub Actions](/guides/gha.md) +- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) +- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) + +### Next steps + +Next, learn how you can locally test and debug your workloads on Kubernetes before deploying. + +## Test your Rust deployment + +### Prerequisites + +- Complete the previous sections of this guide, starting with [Develop your Rust application](develop.md). +- [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. + +### Overview + +In this section, you'll learn how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. This lets you to test and debug your workloads on Kubernetes locally before deploying. + +### Create a Kubernetes YAML file + +In your `docker-rust-postgres` directory, create a file named +`docker-rust-kubernetes.yaml`. Open the file in an IDE or text editor and add +the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker +username and the name of the repository that you created in [Configure CI/CD for +your Rust application](configure-ci-cd.md). + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + labels: + service: server + name: server + namespace: default +spec: + replicas: 1 + selector: + matchLabels: + service: server + strategy: {} + template: + metadata: + labels: + service: server + spec: + initContainers: + - name: wait-for-db + image: busybox:1.28 + command: + [ + "sh", + "-c", + 'until nc -zv db 5432; do echo "waiting for db"; sleep 2; done;', + ] + containers: + - image: DOCKER_USERNAME/REPO_NAME + name: server + imagePullPolicy: Always + ports: + - containerPort: 8000 + hostPort: 5000 + protocol: TCP + env: + - name: ADDRESS + value: 0.0.0.0:8000 + - name: PG_DBNAME + value: example + - name: PG_HOST + value: db + - name: PG_PASSWORD + value: mysecretpassword + - name: PG_USER + value: postgres + - name: RUST_LOG + value: debug + resources: {} + restartPolicy: Always +status: {} +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + labels: + service: db + name: db + namespace: default +spec: + replicas: 1 + selector: + matchLabels: + service: db + strategy: + type: Recreate + template: + metadata: + labels: + service: db + spec: + containers: + - env: + - name: POSTGRES_DB + value: example + - name: POSTGRES_PASSWORD + value: mysecretpassword + - name: POSTGRES_USER + value: postgres + image: postgres:18 + name: db + ports: + - containerPort: 5432 + protocol: TCP + resources: {} + restartPolicy: Always +status: {} +--- +apiVersion: v1 +kind: Service +metadata: + labels: + service: server + name: server + namespace: default +spec: + type: NodePort + ports: + - name: "5000" + port: 5000 + targetPort: 8000 + nodePort: 30001 + selector: + service: server +status: + loadBalancer: {} +--- +apiVersion: v1 +kind: Service +metadata: + labels: + service: db + name: db + namespace: default +spec: + ports: + - name: "5432" + port: 5432 + targetPort: 5432 + selector: + service: db +status: + loadBalancer: {} +``` + +In this Kubernetes YAML file, there are four objects, separated by the `---`. In addition to a Service and Deployment for the database, the other two objects are: + +- A Deployment, describing a scalable group of identical pods. In this case, + you'll get just one replica, or copy of your pod. That pod, which is + described under `template`, has just one container in it. The container is + created from the image built by GitHub Actions in [Configure CI/CD for your + Rust application](configure-ci-cd.md). +- A NodePort service, which will route traffic from port 30001 on your host to + port 5000 inside the pods it routes to, allowing you to reach your app + from the network. + +To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). + +### Deploy and check your application + +1. In a terminal, navigate to `docker-rust-postgres` and deploy your application + to Kubernetes. + + ```console + $ kubectl apply -f docker-rust-kubernetes.yaml + ``` + + You should see output that looks like the following, indicating your Kubernetes objects were created successfully. + + ```shell + deployment.apps/server created + deployment.apps/db created + service/server created + service/db created + ``` + +2. Make sure everything worked by listing your deployments. + + ```console + $ kubectl get deployments + ``` + + Your deployment should be listed as follows: + + ```shell + NAME READY UP-TO-DATE AVAILABLE AGE + db 1/1 1 1 2m21s + server 1/1 1 1 2m21s + ``` + + This indicates all of the pods you asked for in your YAML are up and running. Do the same check for your services. + + ```console + $ kubectl get services + ``` + + You should get output like the following. + + ```shell + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + db ClusterIP 10.105.167.81 5432/TCP 109s + kubernetes ClusterIP 10.96.0.1 443/TCP 9d + server NodePort 10.101.235.213 5000:30001/TCP 109s + ``` + + In addition to the default `kubernetes` service, you can see your `service-entrypoint` service, accepting traffic on port 30001/TCP. + +3. In a terminal, curl the service. + + ```console + $ curl http://localhost:30001/users + [{"id":1,"login":"root"}] + ``` + +4. Run the following command to tear down your application. + + ```console + $ kubectl delete -f docker-rust-kubernetes.yaml + ``` + +### Summary + +In this section, you learned how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. + +Related information: + +- [Kubernetes documentation](https://kubernetes.io/docs/home/) +- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) +- [Swarm mode overview](/manuals/engine/swarm/_index.md) diff --git a/content/guides/rust/build-images.md b/content/guides/rust/build-images.md deleted file mode 100644 index 013dff8ff2be..000000000000 --- a/content/guides/rust/build-images.md +++ /dev/null @@ -1,310 +0,0 @@ ---- -title: Build your Rust image -linkTitle: Build images -weight: 5 -keywords: rust, build, images, dockerfile -description: Learn how to build your first Rust Docker image -aliases: - - /language/rust/build-images/ - - /guides/language/rust/build-images/ ---- - -## Prerequisites - -- You have installed the latest version of [Docker Desktop](/get-started/get-docker.md). -- You have a [git client](https://git-scm.com/downloads). The examples in this section use a command-line based git client, but you can use any client. - -## Overview - -This guide walks you through building your first Rust image. An image -includes everything needed to run an application - the code or binary, runtime, -dependencies, and any other file system objects required. - -## Get the sample application - -Clone the sample application to use with this guide. Open a terminal, change directory to a directory that you want to work in, and run the following command to clone the repository: - -```console -$ git clone https://github.com/docker/docker-rust-hello && cd docker-rust-hello -``` - -## Choose a base image - -> [!TIP] -> -> [Gordon](/ai/gordon/), Docker's AI assistant, can generate Docker assets for your project. Ask Gordon to create a Dockerfile, Compose file, and `.dockerignore` tailored to your application. - -Before editing your Dockerfile, you need to choose a base image. You can use the [Rust Docker Official Image](https://hub.docker.com/_/rust), -or a [Docker Hardened Image (DHI)](https://hub.docker.com/hardened-images/catalog/dhi/rust). - -Docker Hardened Images (DHIs) are minimal, secure, and production-ready base images maintained by Docker. -They help reduce vulnerabilities and simplify compliance. For more details, see [Docker Hardened Images](/dhi/). - -{{< tabs >}} -{{< tab name="Using Docker Hardened Images" >}} - -Docker Hardened Images (DHIs) are publicly available and can be used directly as base images. -To pull Docker Hardened Images, authenticate once with Docker: - -```bash -docker login dhi.io -``` - -Use DHIs from the dhi.io registry, for example: - -```bash -FROM dhi.io/rust:${RUST_VERSION}-alpine3.22-dev AS build -``` - -The following Dockerfile uses a Rust DHI as the build base image: - -```dockerfile {title=Dockerfile} -# Make sure RUST_VERSION matches the Rust version -ARG RUST_VERSION=1.92 -ARG APP_NAME=docker-rust-hello - -################################################################################ -# Create a stage for building the application. -################################################################################ - -FROM dhi.io/rust:${RUST_VERSION}-alpine3.22-dev AS build -ARG APP_NAME -WORKDIR /app - -# Install host build dependencies. -RUN apk add --no-cache clang lld musl-dev git - -# Build the application. -RUN --mount=type=bind,source=src,target=src \ - --mount=type=bind,source=Cargo.toml,target=Cargo.toml \ - --mount=type=bind,source=Cargo.lock,target=Cargo.lock \ - --mount=type=cache,target=/app/target/ \ - --mount=type=cache,target=/usr/local/cargo/git/db \ - --mount=type=cache,target=/usr/local/cargo/registry/ \ - cargo build --locked --release && \ - cp ./target/release/$APP_NAME /bin/server - -################################################################################ -# Create a new stage for running the application that contains the minimal -# We use dhi.io/static for the final stage because it’s a minimal Docker Hardened Image runtime (basically “just # enough OS to run the binary”), which helps keep the image small and with a lower attack surface compared to a # # full Alpine/Debian runtime. -################################################################################ - -FROM dhi.io/static:20250419 AS final - -# Copy the executable from the "build" stage. -COPY --from=build /bin/server /bin/ - -# Configure rocket to listen on all interfaces. -ENV ROCKET_ADDRESS=0.0.0.0 - -# Expose the port that the application listens on. -EXPOSE 8000 - -# What the container should run when it is started. -CMD ["/bin/server"] - -``` - -{{< /tab >}} -{{< tab name="Using the Docker Official Images" >}} - -```dockerfile {title=Dockerfile} -# Pin the Rust toolchain version used in the build stage. -ARG RUST_VERSION=1.92 - -# Name of the compiled binary produced by Cargo (must match Cargo.toml package name). -ARG APP_NAME=docker-rust-hello - -################################################################################ -# Build stage (DOI Rust image) -# This stage compiles the application. -################################################################################ - -FROM docker.io/library/rust:${RUST_VERSION}-alpine AS build - -# Re-declare args inside the stage if you want to use them here. -ARG APP_NAME - -# All build steps happen inside /app. -WORKDIR /app - -# Install build dependencies needed to compile Rust crates on Alpine -RUN apk add --no-cache clang lld musl-dev git - -# Build the application -RUN --mount=type=bind,source=src,target=src \ - --mount=type=bind,source=Cargo.toml,target=Cargo.toml \ - --mount=type=bind,source=Cargo.lock,target=Cargo.lock \ - --mount=type=cache,target=/app/target/ \ - --mount=type=cache,target=/usr/local/cargo/git/db \ - --mount=type=cache,target=/usr/local/cargo/registry/ \ - cargo build --locked --release && \ - cp ./target/release/$APP_NAME /bin/server - -################################################################################ -# Runtime stage (DOI Alpine image) -# This stage runs the already-compiled binary with minimal dependencies. -################################################################################ - -FROM docker.io/library/alpine:3.18 AS final - -# Create a non-privileged user (recommended best practice) -ARG UID=10001 -RUN adduser \ - --disabled-password \ - --gecos "" \ - --home "/nonexistent" \ - --shell "/sbin/nologin" \ - --no-create-home \ - --uid "${UID}" \ - appuser - -# Drop privileges for runtime. -USER appuser - -# Copy only the compiled binary from the build stage. -COPY --from=build /bin/server /bin/ - -# Rocket: listen on all interfaces inside the container. -ENV ROCKET_ADDRESS=0.0.0.0 - -# Document the port your app listens on. -EXPOSE 8000 - -# Start the application. -CMD ["/bin/server"] -``` -{{< /tab >}} -{{< /tabs >}} - - - -For building an image, only the Dockerfile is necessary. Open the Dockerfile -in your favorite IDE or text editor and see what it contains. To learn more -about Dockerfiles, see the [Dockerfile reference](/reference/dockerfile.md). - -## .dockerignore file - -The [`.dockerignore`](/reference/dockerfile.md#dockerignore-file) file specifies patterns and paths that you don't want copied into the image in order to keep the image as small as possible. Open up the `.dockerignore` file in your favorite IDE or text editor to review its contents. - -## Build an image - -Now that you’ve created the Dockerfile, you can build the image. To do this, use -the `docker build` command. The `docker build` command builds Docker images from -a Dockerfile and a context. A build's context is the set of files located in -the specified PATH or URL. The Docker build process can access any of the files -located in this context. - -The build command optionally takes a `--tag` flag. The tag sets the name of the -image and an optional tag in the format `name:tag`. If you don't pass a tag, -Docker uses "latest" as its default tag. - -Build the Docker image. - -```console -$ docker build --tag docker-rust-image-dhi . -``` - -You should see output like the following. - -```console -[+] Building 1.4s (13/13) FINISHED docker:desktop-linux - => [internal] load build definition from Dockerfile 0.0s - => => transferring dockerfile: 1.67kB 0.0s - => [internal] load metadata for dhi.io/static:20250419 1.1s - => [internal] load metadata for dhi.io/rust:1.92-alpine3.22-dev 1.2s - => [auth] static:pull token for dhi.io 0.0s - => [auth] rust:pull token for dhi.io 0.0s - => [internal] load .dockerignore 0.0s - => => transferring context: 646B 0.0s - => [build 1/3] FROM dhi.io/rust:1.92-alpine3.22-dev@sha256:49eb72825a9e15fe48f2c4875a63c7e7f52a5b430bb52b8254b91d132aa5bf38 0.0s - => => resolve dhi.io/rust:1.92-alpine3.22-dev@sha256:49eb72825a9e15fe48f2c4875a63c7e7f52a5b430bb52b8254b91d132aa5bf38 0.0s - => [final 1/2] FROM dhi.io/static:20250419@sha256:74fc43fa240887b8159970e434244039aab0c6efaaa9cf044004cdc22aa2a34d 0.0s - => => resolve dhi.io/static:20250419@sha256:74fc43fa240887b8159970e434244039aab0c6efaaa9cf044004cdc22aa2a34d 0.0s - => [internal] load build context 0.0s - => => transferring context: 117B 0.0s - => CACHED [build 2/3] WORKDIR /build 0.0s - => CACHED [build 3/3] RUN --mount=type=bind,source=src,target=src --mount=type=bind,source=Cargo.toml,target=Cargo.toml --mount=type=bind,source=Cargo.lock,target=Cargo 0.0s - => CACHED [final 2/2] COPY --from=build /build/target/release/docker-rust-hello /server 0.0s - => exporting to image 0.1s - => => exporting layers 0.0s - => => exporting manifest sha256:cc937bbdd712ef6e5445501f77e02ef8455ef64c567598786d46b7b21a4d4fa8 0.0s - => => exporting config sha256:077507b483af4b5e1a928e527e4bb3a4aaf0557e1eea81cd39465f564c187669 0.0s - => => exporting attestation manifest sha256:11b60e7608170493da1fdd88c120e2d2957f2a72a22edbc9cfbdd0dd37d21f89 0.0s - => => exporting manifest list sha256:99a1b925a8d6ebf80e376b8a1e50cd806ec42d194479a3375e1cd9d2911b4db9 0.0s - => => naming to docker.io/library/docker-rust-image-dhi:latest 0.0s - => => unpacking to docker.io/library/docker-rust-image-dhi:latest 0.0s - -View build details: docker-desktop://dashboard/build/desktop-linux/desktop-linux/yczk0ijw8kc5g20e8nbc8r6lj -``` - -## View local images - -To see a list of images you have on your local machine, you have two options. One is to use the Docker CLI and the other is to use [Docker Desktop](/manuals/desktop/use-desktop/images.md). As you are working in the terminal already, take a look at listing images using the CLI. - -To list images, run the `docker images` command. - -```console -$ docker images -IMAGE ID DISK USAGE CONTENT SIZE EXTRA -docker-rust-image-dhi:latest 99a1b925a8d6 11.6MB 2.45MB U -``` - -You should see at least one image listed, including the image you just built `docker-rust-image-dhi:latest`. - -## Tag images - -As mentioned earlier, an image name is made up of slash-separated name components. Name components may contain lowercase letters, digits, and separators. A separator can include a period, one or two underscores, or one or more dashes. A name component may not start or end with a separator. - -An image is made up of a manifest and a list of layers. Don't worry too much about manifests and layers at this point other than a "tag" points to a combination of these artifacts. You can have multiple tags for an image. Create a second tag for the image you built and take a look at its layers. - -To create a new tag for the image you built, run the following command. - -```console -$ docker tag docker-rust-image-dhi:latest docker-rust-image-dhi:v1.0.0 -``` - -The `docker tag` command creates a new tag for an image. It doesn't create a new image. The tag points to the same image and is just another way to reference the image. - -Now, run the `docker images` command to see a list of the local images. - -```console -$ docker images -IMAGE ID DISK USAGE CONTENT SIZE EXTRA -docker-rust-image-dhi:latest 99a1b925a8d6 11.6MB 2.45MB U -docker-rust-image-dhi:v1.0.0 99a1b925a8d6 11.6MB 2.45MB U -``` - -You can see that two images start with `docker-rust-image-dhi`. You know they're the same image because if you take a look at the `IMAGE ID` column, you can see that the values are the same for the two images. - -Remove the tag you just created. To do this, use the `rmi` command. The `rmi` command stands for remove image. - -```console -$ docker rmi docker-rust-image-dhi:v1.0.0 -Untagged: docker-rust-image-dhi:v1.0.0 -``` - -Note that the response from Docker tells you that Docker didn't remove the image, but only "untagged" it. You can check this by running the `docker images` command. - -```console -$ docker images -IMAGE ID DISK USAGE CONTENT SIZE EXTRA -docker-rust-image-dhi:latest 99a1b925a8d6 11.6MB 2.45MB U -``` - -Docker removed the image tagged with `:v1.0.0`, but the `docker-rust-image-dhi:latest` tag is available on your machine. - -## Summary - -This section showed how to create a Dockerfile and `.dockerignore` file for a Rust application, build an image, and tag and list images. - -Related information: - -- [Dockerfile reference](/reference/dockerfile.md) -- [.dockerignore file](/reference/dockerfile.md#dockerignore-file) -- [docker build CLI reference](/reference/cli/docker/buildx/build/) -- [Docker Hardened Images](/dhi/) - -## Next steps - -In the next section learn how to run your image as a container. diff --git a/content/guides/rust/configure-ci-cd.md b/content/guides/rust/configure-ci-cd.md deleted file mode 100644 index 80a6f287c715..000000000000 --- a/content/guides/rust/configure-ci-cd.md +++ /dev/null @@ -1,132 +0,0 @@ ---- -title: Configure CI/CD for your Rust application -linkTitle: Configure CI/CD -weight: 40 -keywords: rust, CI/CD, local, development -description: Learn how to Configure CI/CD for your application -aliases: - - /language/rust/configure-ci-cd/ - - /guides/language/rust/configure-ci-cd/ ---- - -## Prerequisites - -Complete the previous sections of this guide, starting with [Develop your Rust application](develop.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. - -## Overview - -In this section, you'll learn how to set up and use GitHub Actions to build and push your Docker image to Docker Hub. You will complete the following steps: - -1. Create a new repository on GitHub. -2. Define the GitHub Actions workflow. -3. Run the workflow. - -## Step one: Create the repository - -Create a GitHub repository, configure the Docker Hub credentials, and push your source code. - -1. [Create a new repository](https://github.com/new) on GitHub. - -2. Open the repository **Settings**, and go to **Secrets and variables** > - **Actions**. - -3. Create a new **Repository variable** named `DOCKER_USERNAME` and your Docker ID as a value. - -4. Create a new [Personal Access Token (PAT)](/manuals/security/access-tokens.md#create-an-access-token) for Docker Hub. You can name this token `docker-tutorial`. Make sure access permissions include Read and Write. - -5. Add the PAT as a **Repository secret** in your GitHub repository, with the name - `DOCKERHUB_TOKEN`. - -6. In your local repository on your machine, run the following command to change - the origin to the repository you just created. Make sure you change - `your-username` to your GitHub username and `your-repository` to the name of - the repository you created. - - ```console - $ git remote set-url origin https://github.com/your-username/your-repository.git - ``` - -7. Run the following commands to stage, commit, and push your local repository to GitHub. - - ```console - $ git add -A - $ git commit -m "my commit" - $ git push -u origin main - ``` - -## Step two: Set up the workflow - -Set up your GitHub Actions workflow for building, testing, and pushing the image -to Docker Hub. - -1. Go to your repository on GitHub and then select the **Actions** tab. - -2. Select **set up a workflow yourself**. - - This takes you to a page for creating a new GitHub actions workflow file in - your repository, under `.github/workflows/main.yml` by default. - -3. In the editor window, copy and paste the following YAML configuration. - - ```yaml - name: ci - - on: - push: - branches: - - main - - jobs: - build: - runs-on: ubuntu-latest - steps: - - name: Login to Docker Hub - uses: docker/login-action@{{% param "login_action_version" %}} - with: - username: ${{ vars.DOCKER_USERNAME }} - password: ${{ secrets.DOCKERHUB_TOKEN }} - - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} - - - name: Build and push - uses: docker/build-push-action@{{% param "build_push_action_version" %}} - with: - push: true - tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest - ``` - - For more information about the YAML syntax for `docker/build-push-action`, - refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). - -## Step three: Run the workflow - -Save the workflow file and run the job. - -1. Select **Commit changes...** and push the changes to the `main` branch. - - After pushing the commit, the workflow starts automatically. - -2. Go to the **Actions** tab. It displays the workflow. - - Selecting the workflow shows you the breakdown of all the steps. - -3. When the workflow is complete, go to your - [repositories on Docker Hub](https://hub.docker.com/repositories). - - If you see the new repository in that list, it means the GitHub Actions - successfully pushed the image to Docker Hub. - -## Summary - -In this section, you learned how to set up a GitHub Actions workflow for your Rust application. - -Related information: - -- [Introduction to GitHub Actions](/guides/gha.md) -- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) -- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) - -## Next steps - -Next, learn how you can locally test and debug your workloads on Kubernetes before deploying. diff --git a/content/guides/rust/deploy.md b/content/guides/rust/deploy.md deleted file mode 100644 index a43551f673c6..000000000000 --- a/content/guides/rust/deploy.md +++ /dev/null @@ -1,238 +0,0 @@ ---- -title: Test your Rust deployment -linkTitle: Test your deployment -weight: 50 -keywords: deploy, kubernetes, rust -description: Learn how to test your Rust deployment locally using Kubernetes -aliases: - - /language/rust/deploy/ - - /guides/language/rust/deploy/ ---- - -## Prerequisites - -- Complete the previous sections of this guide, starting with [Develop your Rust application](develop.md). -- [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. - -## Overview - -In this section, you'll learn how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. This lets you to test and debug your workloads on Kubernetes locally before deploying. - -## Create a Kubernetes YAML file - -In your `docker-rust-postgres` directory, create a file named -`docker-rust-kubernetes.yaml`. Open the file in an IDE or text editor and add -the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker -username and the name of the repository that you created in [Configure CI/CD for -your Rust application](configure-ci-cd.md). - -```yaml -apiVersion: apps/v1 -kind: Deployment -metadata: - labels: - service: server - name: server - namespace: default -spec: - replicas: 1 - selector: - matchLabels: - service: server - strategy: {} - template: - metadata: - labels: - service: server - spec: - initContainers: - - name: wait-for-db - image: busybox:1.28 - command: - [ - "sh", - "-c", - 'until nc -zv db 5432; do echo "waiting for db"; sleep 2; done;', - ] - containers: - - image: DOCKER_USERNAME/REPO_NAME - name: server - imagePullPolicy: Always - ports: - - containerPort: 8000 - hostPort: 5000 - protocol: TCP - env: - - name: ADDRESS - value: 0.0.0.0:8000 - - name: PG_DBNAME - value: example - - name: PG_HOST - value: db - - name: PG_PASSWORD - value: mysecretpassword - - name: PG_USER - value: postgres - - name: RUST_LOG - value: debug - resources: {} - restartPolicy: Always -status: {} ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - labels: - service: db - name: db - namespace: default -spec: - replicas: 1 - selector: - matchLabels: - service: db - strategy: - type: Recreate - template: - metadata: - labels: - service: db - spec: - containers: - - env: - - name: POSTGRES_DB - value: example - - name: POSTGRES_PASSWORD - value: mysecretpassword - - name: POSTGRES_USER - value: postgres - image: postgres:18 - name: db - ports: - - containerPort: 5432 - protocol: TCP - resources: {} - restartPolicy: Always -status: {} ---- -apiVersion: v1 -kind: Service -metadata: - labels: - service: server - name: server - namespace: default -spec: - type: NodePort - ports: - - name: "5000" - port: 5000 - targetPort: 8000 - nodePort: 30001 - selector: - service: server -status: - loadBalancer: {} ---- -apiVersion: v1 -kind: Service -metadata: - labels: - service: db - name: db - namespace: default -spec: - ports: - - name: "5432" - port: 5432 - targetPort: 5432 - selector: - service: db -status: - loadBalancer: {} -``` - -In this Kubernetes YAML file, there are four objects, separated by the `---`. In addition to a Service and Deployment for the database, the other two objects are: - -- A Deployment, describing a scalable group of identical pods. In this case, - you'll get just one replica, or copy of your pod. That pod, which is - described under `template`, has just one container in it. The container is - created from the image built by GitHub Actions in [Configure CI/CD for your - Rust application](configure-ci-cd.md). -- A NodePort service, which will route traffic from port 30001 on your host to - port 5000 inside the pods it routes to, allowing you to reach your app - from the network. - -To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). - -## Deploy and check your application - -1. In a terminal, navigate to `docker-rust-postgres` and deploy your application - to Kubernetes. - - ```console - $ kubectl apply -f docker-rust-kubernetes.yaml - ``` - - You should see output that looks like the following, indicating your Kubernetes objects were created successfully. - - ```shell - deployment.apps/server created - deployment.apps/db created - service/server created - service/db created - ``` - -2. Make sure everything worked by listing your deployments. - - ```console - $ kubectl get deployments - ``` - - Your deployment should be listed as follows: - - ```shell - NAME READY UP-TO-DATE AVAILABLE AGE - db 1/1 1 1 2m21s - server 1/1 1 1 2m21s - ``` - - This indicates all of the pods you asked for in your YAML are up and running. Do the same check for your services. - - ```console - $ kubectl get services - ``` - - You should get output like the following. - - ```shell - NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE - db ClusterIP 10.105.167.81 5432/TCP 109s - kubernetes ClusterIP 10.96.0.1 443/TCP 9d - server NodePort 10.101.235.213 5000:30001/TCP 109s - ``` - - In addition to the default `kubernetes` service, you can see your `service-entrypoint` service, accepting traffic on port 30001/TCP. - -3. In a terminal, curl the service. - - ```console - $ curl http://localhost:30001/users - [{"id":1,"login":"root"}] - ``` - -4. Run the following command to tear down your application. - - ```console - $ kubectl delete -f docker-rust-kubernetes.yaml - ``` - -## Summary - -In this section, you learned how to use Docker Desktop to deploy your application to a fully-featured Kubernetes environment on your development machine. - -Related information: - -- [Kubernetes documentation](https://kubernetes.io/docs/home/) -- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) -- [Swarm mode overview](/manuals/engine/swarm/_index.md) diff --git a/content/guides/rust/develop.md b/content/guides/rust/develop.md deleted file mode 100644 index b3cbdf0198ec..000000000000 --- a/content/guides/rust/develop.md +++ /dev/null @@ -1,303 +0,0 @@ ---- -title: Develop your Rust application -linkTitle: Develop your app -weight: 20 -keywords: rust, local, development, run, -description: Learn how to develop your Rust application locally. -aliases: - - /language/rust/develop/ - - /guides/language/rust/develop/ ---- - -## Prerequisites - -- You have installed the latest version of [Docker Desktop](/get-started/get-docker.md). -- You have completed the walkthroughs in the Docker Desktop [Learning Center](/manuals/desktop/use-desktop/_index.md) to learn about Docker concepts. -- You have a [git client](https://git-scm.com/downloads). The examples in this section use a command-line based git client, but you can use any client. - -## Overview - -In this section, you’ll learn how to use volumes and networking in Docker. You’ll also use Docker to build your images and Docker Compose to make everything a whole lot easier. - -First, you’ll take a look at running a database in a container and how you can use volumes and networking to persist your data and let your application talk with the database. Then you’ll pull everything together into a Compose file which lets you set up and run a local development environment with one command. - -## Run a database in a container - -Instead of downloading PostgreSQL, installing, configuring, and then running the PostgreSQL database as a service, you can use the Docker Official Image for PostgreSQL and run it in a container. - -Before you run PostgreSQL in a container, create a volume that Docker can manage to store your persistent data and configuration. Use the named volumes feature that Docker provides instead of using bind mounts. - -Run the following command to create your volume. - -```console -$ docker volume create db-data -``` - -Now create a network that your application and database will use to talk to each other. The network is called a user-defined bridge network and gives you a nice DNS lookup service which you can use when creating your connection string. - -```console -$ docker network create postgresnet -``` - -Now you can run PostgreSQL in a container and attach to the volume and network that you created previously. Docker pulls the image from Hub and runs it for you locally. -In the following command, option `--mount` is for starting the container with a volume. For more information, see [Docker volumes](/manuals/engine/storage/volumes.md). - -```console -$ docker run --rm -d --mount \ - "type=volume,src=db-data,target=/var/lib/postgresql" \ - -p 5432:5432 \ - --network postgresnet \ - --name db \ - -e POSTGRES_PASSWORD=mysecretpassword \ - -e POSTGRES_DB=example \ - postgres:18 -``` - -Now, make sure that your PostgreSQL database is running and that you can connect to it. Connect to the running PostgreSQL database inside the container. - -```console -$ docker exec -it db psql -U postgres -``` - -You should see output like the following. - -```console -psql (15.3 (Debian 15.3-1.pgdg110+1)) -Type "help" for help. - -postgres=# -``` - -In the previous command, you logged in to the PostgreSQL database by passing the `psql` command to the `db` container. Press ctrl-d to exit the PostgreSQL interactive terminal. - -## Get and run the sample application - -For the sample application, you'll use a variation of the backend from the react-rust-postgres application from [Awesome Compose](https://github.com/docker/awesome-compose/tree/master/react-rust-postgres). - -1. Clone the sample application repository using the following command. - - ```console - $ git clone https://github.com/docker/docker-rust-postgres - ``` - -2. In the cloned repository's directory, create a `Dockerfile`. This application includes a `migrations` directory (in addition to `src`) to initialize the database, so the Dockerfile includes a bind mount for that directory in the build stage. - - ```dockerfile {hl_lines="28"} - # syntax=docker/dockerfile:1 - - # Comments are provided throughout this file to help you get started. - # If you need more help, visit the Dockerfile reference guide at - # https://docs.docker.com/reference/dockerfile/ - - ################################################################################ - # Create a stage for building the application. - - ARG RUST_VERSION=1.70.0 - ARG APP_NAME=react-rust-postgres - FROM rust:${RUST_VERSION}-slim-bullseye AS build - ARG APP_NAME - WORKDIR /app - - # Build the application. - # Leverage a cache mount to /usr/local/cargo/registry/ - # for downloaded dependencies and a cache mount to /app/target/ for - # compiled dependencies which will speed up subsequent builds. - # Leverage a bind mount to the src directory to avoid having to copy the - # source code into the container. Once built, copy the executable to an - # output directory before the cache mounted /app/target is unmounted. - RUN --mount=type=bind,source=src,target=src \ - --mount=type=bind,source=Cargo.toml,target=Cargo.toml \ - --mount=type=bind,source=Cargo.lock,target=Cargo.lock \ - --mount=type=cache,target=/app/target/ \ - --mount=type=cache,target=/usr/local/cargo/registry/ \ - --mount=type=bind,source=migrations,target=migrations \ - <8000/tcp, [::]:3001->8000/tcp youthful_lamport -``` - -The `docker ps` command provides a bunch of information about your running containers. You can see the container ID, the image running inside the container, the command that was used to start the container, when it was created, the status, ports that were exposed, and the name of the container. - -You are probably wondering where the name of your container is coming from. Since you didn’t provide a name for the container when you started it, Docker generated a random name. You’ll fix this in a minute, but first you need to stop the container. To stop the container, run the `docker stop` command which does just that, stops the container. You need to pass the name of the container or you can use the container ID. - -```console -$ docker stop youthful_lamport -youthful_lamport -``` - -Now, rerun the `docker ps` command to see a list of running containers. - -```console -$ docker ps -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -``` - -## Stop, start, and name containers - -You can start, stop, and restart Docker containers. When you stop a container, it's not removed, but the status is changed to stopped and the process inside the container is stopped. When you ran the `docker ps` command in the previous module, the default output only shows running containers. When you pass the `--all` or `-a` for short, you see all containers on your machine, irrespective of their start or stop status. - -```console -$ docker ps -a -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -3e4830e7f013 docker-rust-image-dhi "/server" About a minute ago Exited (0) 28 seconds ago youthful_lamport -60009b7eaf40 docker-rust-image-dhi "/server" 2 minutes ago Exited (0) About a minute ago sharp_noyce -152e1d7d9eea docker-rust-image-dhi "/server ." 4 minutes ago Exited (0) 2 minutes ago magical_bhabha -``` - -You should now see several containers listed. These are containers that you started and stopped but you haven't removed. - -Restart the container that you just stopped. Locate the name of the container you just stopped and replace the name of the container in following restart command. - -```console -$ docker restart youthful_lamport -``` - -Now list all the containers again using the `docker ps --all` command. - -```console -$ docker ps --all -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -3e4830e7f013 docker-rust-image-dhi "/server" 3 minutes ago Up 7 seconds 0.0.0.0:3001->8000/tcp, [::]:3001->8000/tcp youthful_lamport -60009b7eaf40 docker-rust-image-dhi "/server" 4 minutes ago Exited (0) 3 minutes ago sharp_noyce -152e1d7d9eea docker-rust-image-dhi "/server ." 5 minutes ago Exited (0) 4 minutes ago magical_bhabha -``` - -Notice that the container you just restarted has been started in detached mode. Also, observe the status of the container is "Up X seconds". When you restart a container, it starts with the same flags or commands that it was originally started with. - -Now, stop and remove all of your containers and take a look at fixing the random naming issue. Stop the container you just started. Find the name of your running container and replace the name in the following command with the name of the container on your system. - -```console -$ docker stop youthful_lamport -youthful_lamport -``` - -Now that you have stopped all of your containers, remove them. When you remove a container, it's no longer running, nor is it in the stopped status, but the process inside the container has been stopped and the metadata for the container has been removed. - -To remove a container, run the `docker rm` command with the container name. You can pass multiple container names to the command using a single command. Again, replace the container names in the following command with the container names from your system. - -```console -$ docker rm youthful_lamport friendly_montalcini tender_bose -youthful_lamport -sharp_noyce -magical_bhabha -``` - -Run the `docker ps --all` command again to see that Docker removed all containers. - -Now, it's time to address the random naming issue. Standard practice is to name your containers for the simple reason that it's easier to identify what's running in the container and what application or service it's associated with. - -To name a container, pass the `--name` flag to the `docker run` command. - -```console -$ docker run -d -p 3001:8000 --name docker-rust-container docker-rust-image-dhi -1aa5d46418a68705c81782a58456a4ccdb56a309cb5e6bd399478d01eaa5cdda -$ docker ps -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -219b2e3c7c38 docker-rust-image-dhi "/server" 6 seconds ago Up 5 seconds 0.0.0.0:3001->8000/tcp, [::]:3001->8000/tcp docker-rust-container -``` - -Now you can identify your container based on the name. - -## Summary - -In this section, you took a look at running containers. You also took a look at managing containers by starting, stopping, and restarting them. And finally, you looked at naming your containers so they are more identifiable. - -Related information: - -- [docker run CLI reference](/reference/cli/docker/container/run/) - -## Next steps - -In the next section, you’ll learn how to run a database in a container and connect it to a Rust application. diff --git a/content/guides/sentiment-analysis.md b/content/guides/sentiment-analysis.md index 725899efe9fc..c9883cf0af9e 100644 --- a/content/guides/sentiment-analysis.md +++ b/content/guides/sentiment-analysis.md @@ -6,11 +6,10 @@ description: Learn how to build and run a sentiment analysis application using P summary: | This guide demonstrates how to containerize sentiment analysis models using Docker. -tags: [ai] -languages: [python] aliases: - /guides/use-case/nlp/sentiment-analysis/ params: + tags: [ai] time: 20 minutes --- diff --git a/content/guides/swarm-deploy.md b/content/guides/swarm-deploy.md index ff04acec9554..132e01cc0f3e 100644 --- a/content/guides/swarm-deploy.md +++ b/content/guides/swarm-deploy.md @@ -8,8 +8,8 @@ aliases: - /guides/deployment-orchestration/swarm-deploy/ summary: | Discover how to deploy and manage Docker containers using Docker Swarm. -tags: [deploy] params: + tags: [deployment] time: 10 minutes --- diff --git a/content/guides/tensorflowjs.md b/content/guides/tensorflowjs.md index 6b1a2e398edc..1810de2c11aa 100644 --- a/content/guides/tensorflowjs.md +++ b/content/guides/tensorflowjs.md @@ -4,11 +4,10 @@ keywords: tensorflow.js, machine learning, ml, mediapipe, blazeface, face detect title: Face detection with TensorFlow.js summary: | This guide explains how to run TensorFlow.js in Docker containers. -tags: [ai] -languages: [js] aliases: - /guides/use-case/tensorflowjs/ params: + tags: [ai] time: 20 minutes --- diff --git a/content/guides/testcontainers-cloud/_index.md b/content/guides/testcontainers-cloud/_index.md index 1310bcdf6261..c47b205f4eaa 100644 --- a/content/guides/testcontainers-cloud/_index.md +++ b/content/guides/testcontainers-cloud/_index.md @@ -7,25 +7,19 @@ summary: | description: | Testcontainers Cloud by Docker streamlines integration testing by offloading container management to the cloud. It enables faster, consistent tests for containerized services like databases, improving performance and scalability in CI/CD pipelines without straining local or CI resources. Ideal for developers needing efficient, reliable testing environments. keywords: testcontainers cloud, integration testing, ci/cd, containerized tests, cloud testing, scalable testing -tags: [product-demo] +aliases: + - /guides/testcontainers-cloud/common-questions/ + - /guides/testcontainers-cloud/demo-ci/ + - /guides/testcontainers-cloud/demo-local/ + - /guides/testcontainers-cloud/why/ params: + tags: [testing] image: images/learning-paths/testcontainers-cloud-learning-path.png time: 12 minutes - resource_links: - - title: Testcontainers Guides - url: https://testcontainers.com/guides - - title: Testcontainers Best Practices - url: https://www.docker.com/blog/testcontainers-best-practices/ - - title: Simple local development with Testcontainers Desktop - url: https://testcontainers.com/guides/simple-local-development-with-testcontainers-desktop/ - - title: Streamlining Local Development with Dev Containers and Testcontainers Cloud - url: https://www.docker.com/blog/streamlining-local-development-with-dev-containers-and-testcontainers-cloud/ - - title: Running Testcontainers Tests Using GitHub Actions and Testcontainers Cloud - url: https://www.docker.com/blog/running-testcontainers-tests-using-github-actions/ - - title: Testcontainers Cloud on the Docker Blog url: https://www.docker.com/search/?_sf_s=testcontainers%20cloud --- + Testcontainers Cloud is a cloud-based solution designed to streamline and enhance the process of running integration tests using Testcontainers. Testcontainers is the open source framework, which allows developers to easily spin up containerized dependencies such as databases, message brokers, and other services required for testing. By shifting the management of Testcontainers-based services to the cloud, Testcontainers Cloud optimizes performance, reduces resource constraints on local machines or CI servers, and ensures consistent test environments. This solution is particularly beneficial for teams working on complex, distributed systems, as it allows for scalable, isolated, and reliable testing without the typical overhead of managing containers locally. ## What you'll learn @@ -46,3 +40,94 @@ Docker Pro, Team, and Business subscriptions come with Testcontainers Cloud runt - DevOps Teams that integrate automated container-based testing into CI/CD pipelines for continuous testing. - QA Teams that seek scalable and consistent test environments for comprehensive integration and end-to-end testing. - Developers who need reliable, containerized test environments for testing microservices and databases. + +## Why Testcontainers Cloud? + +{{< youtube-embed "6dRRlk5Vd0E" >}} + +Testcontainers Cloud is a powerful cloud-based solution designed to optimize integration testing with Testcontainers by offloading container management to the cloud. It helps developers and teams overcome the limitations of traditional local and CI-based testing, ensuring consistent environments, faster test execution, and scalable workflows. Whether you're new to Testcontainers or looking to enhance your existing setup, Testcontainers Cloud offers a seamless way to manage containerized tests, improving efficiency and reliability in your development pipeline. + +Testcontainers Cloud provides several benefits: + +- **Offloading to the Cloud:** Frees up local resources by shifting container management to the cloud, keeping your laptop responsive. +- **Consistent Testing Environments:** Ensures that tests run in isolated, reliable environments, reducing inconsistencies across platforms from Dev to CI. +- **Scalability:** Allows running large numbers of containers simultaneously without being limited by local or CI resources. +- **Faster CI/CD Pipelines:** Reduces configuration bottlenecks and speeds up build times by offloading containers to multiple on-demand cloud workers with the Turbo-mode feature. + +Testcontainers Cloud streamlines integration testing by offloading container management to the cloud, ensuring consistent environments and faster test execution resulting in reduced resource strain, making it an essential tool for improving the stability of your Testcontainers-based workflows. + +## Setting up Testcontainers Cloud by Docker + +{{< youtube-embed "7c3xLAG560U" >}} + +This demo shows the process of setting up Testcontainers Cloud by Docker to +work in your local development environment using the Testcontainers Desktop +application. By the end of this walkthrough, you'll have Testcontainers Cloud +by Docker up and running, ready to offload container management from your local +machine to the cloud for more efficient testing. + +- Install and configure Testcontainers Cloud and the CLI to seamlessly integrate with your local development environment. +- Set up and configure the Testcontainers Desktop application to monitor and manage cloud-based containers during local tests. +- Create and run integration tests using Testcontainers that leverage cloud-based container resources. +- Monitor and manage containers efficiently, understanding how Testcontainers Cloud automates cleanup and ensures consistent testing environments. +- Review options for monitoring and troubleshooting in the Testcontainers Cloud Dashboard. + +## Configuring Testcontainers Cloud in the CI Pipeline + +{{< youtube-embed "NlZY9aumKJU" >}} + +This demo shows how Testcontainers Cloud can be seamlessly integrated into a +Continuous Integration (CI) pipeline using GitHub Workflows, providing a +powerful solution for running containerized integration tests without +overloading local or CI runner resources. By leveraging GitHub Actions, +developers can automate the process of spinning up and managing containers for +testing in the cloud, ensuring faster and more reliable test execution. With +just a few configuration steps, including setting up Testcontainers Cloud +authentication and adding it to your workflow, you can offload container +orchestration to the cloud. This approach improves the scalability of your +pipeline, ensures consistency across tests, and simplifies resource management, +making it an ideal solution for modern, containerized development workflows. + +- Understand how to set up a GitHub Actions workflow to automate the build and testing of a project. +- Learn how to configure Testcontainers Cloud within GitHub Actions to offload containerized testing to the cloud, improving efficiency and resource management. +- Explore how Testcontainers Cloud integrates with GitHub workflows to run integration tests that require containerized services, such as databases and message brokers. + +## Common challenges and questions + + + +#### How is Testcontainers Cloud different from the open-source Testcontainers framework? + +While the open-source Testcontainers is a library that provides a lightweight APIs for bootstrapping local development and test dependencies with real services wrapped in Docker containers, Testcontainers Cloud provides a cloud runtime for these containers. This reduces the resource strain on local environments and provides more scalability, especially in CI/CD workflows, that enables consistent Testcontainers experience across the organization. + +#### What types of containers can I run with Testcontainers Cloud? + +Testcontainers Cloud supports any containers you would typically use with the Testcontainers framework, including databases (PostgreSQL, MySQL, MongoDB), message brokers (Kafka, RabbitMQ), and other services required for integration testing. + +#### Do I need to change my existing test code to use Testcontainers Cloud? + +No, you don't need to change your existing test code. Testcontainers Cloud integrates seamlessly with the open-source Testcontainers framework. Once the cloud configuration is set up, it automatically manages the containers in the cloud without requiring code changes. + +#### How do I integrate Testcontainers Cloud into my project? + +To integrate Testcontainers Cloud, you need to install the Testcontainers Desktop app and select run with Testcontainers Cloud option in the menu. In CI you’ll need to add a workflow step that downloads Testcontainers Cloud agent. No code changes are required beyond enabling Cloud runtime via the Testcontainers Desktop app locally or installing Testcontainers Cloud agent in CI. + +#### Can I use Testcontainers Cloud in a CI/CD pipeline? + +Yes, Testcontainers Cloud is designed to work efficiently in CI/CD pipelines. It helps reduce build times and resource bottlenecks by offloading containers that you spin up with Testcontainers library to the cloud, making it a perfect fit for continuous testing environments. + +#### What are the benefits of using Testcontainers Cloud? + +The key benefits include reduced resource usage on local machines and CI servers, scalability (run more containers without performance degradation), consistent testing environments, centralized monitoring, ease of CI configuration with removed security concerns of running Docker-in-Docker or a privileged daemon. + +#### Does Testcontainers Cloud support all programming languages? + +Testcontainers Cloud supports any language that works with the open-source Testcontainers libraries, including Java, Python, Node.js, Go, and others. As long as your project uses Testcontainers, it can be offloaded to Testcontainers Cloud. + +#### How is container cleanup handled in Testcontainers Cloud? + +While Testcontainers library automatically handles container lifecycle management, Testcontainers Cloud manages the allocated cloud worker lifetime. This means that containers are spun up, monitored, and cleaned up after tests are completed by Testcontainers library, and the worker where these containers have being running will be removed automatically after the ~35 min idle period by Testcontainers Cloud. This approach frees developers from manually managing containers and associated cloud resources. + +#### Is there a free tier or pricing model for Testcontainers Cloud? + +Pricing details for Testcontainers Cloud can be found on the [pricing page](https://testcontainers.com/cloud/pricing/). diff --git a/content/guides/testcontainers-cloud/common-questions.md b/content/guides/testcontainers-cloud/common-questions.md deleted file mode 100644 index 549af8e0f159..000000000000 --- a/content/guides/testcontainers-cloud/common-questions.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -title: Common challenges and questions -description: Explore common challenges and questions related to Testcontainers Cloud by Docker. -keywords: testcontainers cloud, faq, integration testing, troubleshooting, cloud testing -weight: 40 ---- - - - -### How is Testcontainers Cloud different from the open-source Testcontainers framework? - -While the open-source Testcontainers is a library that provides a lightweight APIs for bootstrapping local development and test dependencies with real services wrapped in Docker containers, Testcontainers Cloud provides a cloud runtime for these containers. This reduces the resource strain on local environments and provides more scalability, especially in CI/CD workflows, that enables consistent Testcontainers experience across the organization. - -### What types of containers can I run with Testcontainers Cloud? - -Testcontainers Cloud supports any containers you would typically use with the Testcontainers framework, including databases (PostgreSQL, MySQL, MongoDB), message brokers (Kafka, RabbitMQ), and other services required for integration testing. - -### Do I need to change my existing test code to use Testcontainers Cloud? - -No, you don't need to change your existing test code. Testcontainers Cloud integrates seamlessly with the open-source Testcontainers framework. Once the cloud configuration is set up, it automatically manages the containers in the cloud without requiring code changes. - -### How do I integrate Testcontainers Cloud into my project? - -To integrate Testcontainers Cloud, you need to install the Testcontainers Desktop app and select run with Testcontainers Cloud option in the menu. In CI you’ll need to add a workflow step that downloads Testcontainers Cloud agent. No code changes are required beyond enabling Cloud runtime via the Testcontainers Desktop app locally or installing Testcontainers Cloud agent in CI. - -### Can I use Testcontainers Cloud in a CI/CD pipeline? - -Yes, Testcontainers Cloud is designed to work efficiently in CI/CD pipelines. It helps reduce build times and resource bottlenecks by offloading containers that you spin up with Testcontainers library to the cloud, making it a perfect fit for continuous testing environments. - -### What are the benefits of using Testcontainers Cloud? - -The key benefits include reduced resource usage on local machines and CI servers, scalability (run more containers without performance degradation), consistent testing environments, centralized monitoring, ease of CI configuration with removed security concerns of running Docker-in-Docker or a privileged daemon. - -### Does Testcontainers Cloud support all programming languages? - -Testcontainers Cloud supports any language that works with the open-source Testcontainers libraries, including Java, Python, Node.js, Go, and others. As long as your project uses Testcontainers, it can be offloaded to Testcontainers Cloud. - -### How is container cleanup handled in Testcontainers Cloud? - -While Testcontainers library automatically handles container lifecycle management, Testcontainers Cloud manages the allocated cloud worker lifetime. This means that containers are spun up, monitored, and cleaned up after tests are completed by Testcontainers library, and the worker where these containers have being running will be removed automatically after the ~35 min idle period by Testcontainers Cloud. This approach frees developers from manually managing containers and associated cloud resources. - -### Is there a free tier or pricing model for Testcontainers Cloud? - -Pricing details for Testcontainers Cloud can be found on the [pricing page](https://testcontainers.com/cloud/pricing/). diff --git a/content/guides/testcontainers-cloud/demo-ci.md b/content/guides/testcontainers-cloud/demo-ci.md deleted file mode 100644 index 71cdc625f093..000000000000 --- a/content/guides/testcontainers-cloud/demo-ci.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -title: Configuring Testcontainers Cloud in the CI Pipeline -description: Use Testcontainers Cloud with GitHub Workflows to automate testing in a CI pipeline. -keywords: testcontainers cloud, ci/cd, github actions, integration testing, cloud testing -weight: 30 ---- - -{{< youtube-embed "NlZY9aumKJU" >}} - -This demo shows how Testcontainers Cloud can be seamlessly integrated into a -Continuous Integration (CI) pipeline using GitHub Workflows, providing a -powerful solution for running containerized integration tests without -overloading local or CI runner resources. By leveraging GitHub Actions, -developers can automate the process of spinning up and managing containers for -testing in the cloud, ensuring faster and more reliable test execution. With -just a few configuration steps, including setting up Testcontainers Cloud -authentication and adding it to your workflow, you can offload container -orchestration to the cloud. This approach improves the scalability of your -pipeline, ensures consistency across tests, and simplifies resource management, -making it an ideal solution for modern, containerized development workflows. - -- Understand how to set up a GitHub Actions workflow to automate the build and testing of a project. -- Learn how to configure Testcontainers Cloud within GitHub Actions to offload containerized testing to the cloud, improving efficiency and resource management. -- Explore how Testcontainers Cloud integrates with GitHub workflows to run integration tests that require containerized services, such as databases and message brokers. diff --git a/content/guides/testcontainers-cloud/demo-local.md b/content/guides/testcontainers-cloud/demo-local.md deleted file mode 100644 index 870e4e0be783..000000000000 --- a/content/guides/testcontainers-cloud/demo-local.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -title: Setting up Testcontainers Cloud by Docker -description: Set up Testcontainers Cloud by Docker in a local development environment. -keywords: testcontainers cloud, local development, testcontainers desktop, integration testing, setup -weight: 20 ---- - -{{< youtube-embed "7c3xLAG560U" >}} - -This demo shows the process of setting up Testcontainers Cloud by Docker to -work in your local development environment using the Testcontainers Desktop -application. By the end of this walkthrough, you'll have Testcontainers Cloud -by Docker up and running, ready to offload container management from your local -machine to the cloud for more efficient testing. - -- Install and configure Testcontainers Cloud and the CLI to seamlessly integrate with your local development environment. -- Set up and configure the Testcontainers Desktop application to monitor and manage cloud-based containers during local tests. -- Create and run integration tests using Testcontainers that leverage cloud-based container resources. -- Monitor and manage containers efficiently, understanding how Testcontainers Cloud automates cleanup and ensures consistent testing environments. -- Review options for monitoring and troubleshooting in the Testcontainers Cloud Dashboard. diff --git a/content/guides/testcontainers-cloud/why.md b/content/guides/testcontainers-cloud/why.md deleted file mode 100644 index e09cbfbf7d0d..000000000000 --- a/content/guides/testcontainers-cloud/why.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -title: Why Testcontainers Cloud? -description: Learn how Testcontainers Cloud by Docker can help you optimize integration testing. -keywords: testcontainers cloud, integration testing, scalable testing, ci/cd, cloud testing -weight: 10 ---- - -{{< youtube-embed "6dRRlk5Vd0E" >}} - -Testcontainers Cloud is a powerful cloud-based solution designed to optimize integration testing with Testcontainers by offloading container management to the cloud. It helps developers and teams overcome the limitations of traditional local and CI-based testing, ensuring consistent environments, faster test execution, and scalable workflows. Whether you're new to Testcontainers or looking to enhance your existing setup, Testcontainers Cloud offers a seamless way to manage containerized tests, improving efficiency and reliability in your development pipeline. - -Testcontainers Cloud provides several benefits: - -- **Offloading to the Cloud:** Frees up local resources by shifting container management to the cloud, keeping your laptop responsive. -- **Consistent Testing Environments:** Ensures that tests run in isolated, reliable environments, reducing inconsistencies across platforms from Dev to CI. -- **Scalability:** Allows running large numbers of containers simultaneously without being limited by local or CI resources. -- **Faster CI/CD Pipelines:** Reduces configuration bottlenecks and speeds up build times by offloading containers to multiple on-demand cloud workers with the Turbo-mode feature. - -Testcontainers Cloud streamlines integration testing by offloading container management to the cloud, ensuring consistent environments and faster test execution resulting in reduced resource strain, making it an essential tool for improving the stability of your Testcontainers-based workflows. diff --git a/content/guides/testcontainers-dotnet-aspnet-core/_index.md b/content/guides/testcontainers-dotnet-aspnet-core/_index.md index 92480b54d167..31769970ae01 100644 --- a/content/guides/testcontainers-dotnet-aspnet-core/_index.md +++ b/content/guides/testcontainers-dotnet-aspnet-core/_index.md @@ -6,14 +6,16 @@ keywords: testcontainers, dotnet, csharp, testing, mssql, asp.net core, integrat summary: | Learn how to test an ASP.NET Core web app using Testcontainers for .NET with a real Microsoft SQL Server instance instead of SQLite. -toc_min: 1 -toc_max: 2 -tags: [testing-with-docker] -languages: [c-sharp] +aliases: + - /guides/testcontainers-dotnet-aspnet-core/create-project/ + - /guides/testcontainers-dotnet-aspnet-core/run-tests/ + - /guides/testcontainers-dotnet-aspnet-core/write-tests/ params: + tags: [testing] time: 25 minutes --- + In this guide, you'll learn how to: @@ -34,3 +36,466 @@ In this guide, you'll learn how to: > If you're new to Testcontainers, visit the > [Testcontainers overview](https://testcontainers.com/getting-started/) to learn more about > Testcontainers and the benefits of using it. + +## Set up the project + +### Background + +This guide builds on top of Microsoft's +[Integration tests in ASP.NET Core](https://learn.microsoft.com/en-us/aspnet/core/test/integration-tests) +documentation. The original sample uses an in-memory SQLite database as the +backing store for integration tests. You'll replace SQLite with a real +Microsoft SQL Server instance running in a Docker container using +Testcontainers. + +You can find the original code sample in the +[dotnet/AspNetCore.Docs.Samples](https://github.com/dotnet/AspNetCore.Docs.Samples/tree/main/test/integration-tests/IntegrationTestsSample) +repository. + +### Clone the repository + +Clone the Testcontainers guide repository and change into the project +directory: + +```console +$ git clone https://github.com/testcontainers/tc-guide-testing-aspnet-core.git +$ cd tc-guide-testing-aspnet-core +``` + +### Project structure + +The solution contains two projects: + +```text +RazorPagesProject.sln +├── src/RazorPagesProject/ # ASP.NET Core Razor Pages app +└── tests/RazorPagesProject.Tests/ # xUnit integration tests +``` + +#### Application project + +The application project (`src/RazorPagesProject/RazorPagesProject.csproj`) +is a Razor Pages web app that uses Entity Framework Core with SQLite as its +default database provider: + +```xml + + + + net9.0 + enable + + + + + + + + + all + runtime; build; native; contentfiles; analyzers; buildtransitive + + + + +``` + +The `ApplicationDbContext` stores `Message` entities and provides methods to +query and manage them: + +```csharp +public class ApplicationDbContext : IdentityDbContext +{ + public ApplicationDbContext(DbContextOptions options) + : base(options) + { + } + + public virtual DbSet Messages { get; set; } + + public async virtual Task> GetMessagesAsync() + { + return await Messages + .OrderBy(message => message.Text) + .AsNoTracking() + .ToListAsync(); + } + + public async virtual Task AddMessageAsync(Message message) + { + await Messages.AddAsync(message); + await SaveChangesAsync(); + } + + public async virtual Task DeleteAllMessagesAsync() + { + foreach (Message message in Messages) + { + Messages.Remove(message); + } + + await SaveChangesAsync(); + } + + public async virtual Task DeleteMessageAsync(int id) + { + var message = await Messages.FindAsync(id); + + if (message != null) + { + Messages.Remove(message); + await SaveChangesAsync(); + } + } + + public void Initialize() + { + Messages.AddRange(GetSeedingMessages()); + SaveChanges(); + } + + public static List GetSeedingMessages() + { + return new List() + { + new Message(){ Text = "You're standing on my scarf." }, + new Message(){ Text = "Would you like a jelly baby?" }, + new Message(){ Text = "To the rational mind, nothing is inexplicable; only unexplained." } + }; + } +} +``` + +#### Test project + +The test project (`tests/RazorPagesProject.Tests/RazorPagesProject.Tests.csproj`) +includes xUnit, the ASP.NET Core testing infrastructure, and the +Testcontainers MSSQL module: + +```xml + + + + net9.0 + enable + + + + + + + + + + + + + all + runtime; build; native; contentfiles; analyzers; buildtransitive + + + + + + + + all + runtime; build; native; contentfiles; analyzers; buildtransitive + + + + + + + + + + Always + + + + +``` + +The key dependencies are: + +- `Microsoft.AspNetCore.Mvc.Testing` - provides `WebApplicationFactory` for + bootstrapping the app in tests +- `Microsoft.EntityFrameworkCore.SqlServer` - the SQL Server database provider + for Entity Framework Core +- `Testcontainers.MsSql` - the Testcontainers module for Microsoft SQL Server + +#### Existing SQLite-based test factory + +The original project includes a `CustomWebApplicationFactory` that replaces +the application's database with an in-memory SQLite instance: + +```csharp +public class CustomWebApplicationFactory + : WebApplicationFactory where TProgram : class +{ + protected override void ConfigureWebHost(IWebHostBuilder builder) + { + builder.ConfigureServices(services => + { + var dbContextDescriptor = services.SingleOrDefault( + d => d.ServiceType == + typeof(DbContextOptions)); + + services.Remove(dbContextDescriptor); + + var dbConnectionDescriptor = services.SingleOrDefault( + d => d.ServiceType == + typeof(DbConnection)); + + services.Remove(dbConnectionDescriptor); + + // Create open SqliteConnection so EF won't automatically close it. + services.AddSingleton(container => + { + var connection = new SqliteConnection("DataSource=:memory:"); + connection.Open(); + + return connection; + }); + + services.AddDbContext((container, options) => + { + var connection = container.GetRequiredService(); + options.UseSqlite(connection); + }); + }); + + builder.UseEnvironment("Development"); + } +} +``` + +While this approach works, SQLite has behavioral differences from the database +you'd use in production. In the next section, you'll replace it with a +Testcontainers-managed Microsoft SQL Server instance. + +## Write tests with Testcontainers + +The existing tests use an in-memory SQLite database. While convenient, this +doesn't match production behavior. You can replace it with a real Microsoft SQL +Server instance managed by Testcontainers. + +### Add dependencies + +Change to the test project directory and add the SQL Server Entity Framework +provider and the Testcontainers MSSQL module: + +```console +$ cd tests/RazorPagesProject.Tests +$ dotnet add package Microsoft.EntityFrameworkCore.SqlServer --version 7.0.0 +$ dotnet add package Testcontainers.MsSql --version 3.0.0 +``` + +> [!NOTE] +> Testcontainers for .NET offers a range of +> [modules](https://www.nuget.org/profiles/Testcontainers) that follow best +> practice configurations. + +### Create the test class + +Create a `MsSqlTests.cs` file in the `IntegrationTests` directory. This class +manages the SQL Server container lifecycle and contains a nested test class. + +```csharp +using System.Data.Common; +using System.Net; +using AngleSharp.Html.Dom; +using Microsoft.AspNetCore.Mvc.Testing; +using Microsoft.EntityFrameworkCore; +using RazorPagesProject.Data; +using RazorPagesProject.Tests.Helpers; +using Testcontainers.MsSql; +using Xunit; + +namespace RazorPagesProject.Tests.IntegrationTests; + +public sealed class MsSqlTests : IAsyncLifetime +{ + private readonly MsSqlContainer _msSqlContainer = new MsSqlBuilder().Build(); + + public Task InitializeAsync() + { + return _msSqlContainer.StartAsync(); + } + + public Task DisposeAsync() + { + return _msSqlContainer.DisposeAsync().AsTask(); + } + + public sealed class IndexPageTests : IClassFixture, IDisposable + { + private readonly WebApplicationFactory _webApplicationFactory; + + private readonly HttpClient _httpClient; + + public IndexPageTests(MsSqlTests fixture) + { + var clientOptions = new WebApplicationFactoryClientOptions(); + clientOptions.AllowAutoRedirect = false; + + _webApplicationFactory = new CustomWebApplicationFactory(fixture); + _httpClient = _webApplicationFactory.CreateClient(clientOptions); + } + + public void Dispose() + { + _webApplicationFactory.Dispose(); + } + + [Fact] + public async Task Post_DeleteAllMessagesHandler_ReturnsRedirectToRoot() + { + // Arrange + var defaultPage = await _httpClient.GetAsync("/") + .ConfigureAwait(false); + + var document = await HtmlHelpers.GetDocumentAsync(defaultPage) + .ConfigureAwait(false); + + // Act + var form = (IHtmlFormElement)document.QuerySelector("form[id='messages']"); + var submitButton = (IHtmlButtonElement)document.QuerySelector("button[id='deleteAllBtn']"); + + var response = await _httpClient.SendAsync(form, submitButton) + .ConfigureAwait(false); + + // Assert + Assert.Equal(HttpStatusCode.OK, defaultPage.StatusCode); + Assert.Equal(HttpStatusCode.Redirect, response.StatusCode); + Assert.Equal("/", response.Headers.Location.OriginalString); + } + + private sealed class CustomWebApplicationFactory : WebApplicationFactory + { + private readonly string _connectionString; + + public CustomWebApplicationFactory(MsSqlTests fixture) + { + _connectionString = fixture._msSqlContainer.GetConnectionString(); + } + + protected override void ConfigureWebHost(IWebHostBuilder builder) + { + builder.ConfigureServices(services => + { + services.Remove(services.SingleOrDefault(service => typeof(DbContextOptions) == service.ServiceType)); + services.Remove(services.SingleOrDefault(service => typeof(DbConnection) == service.ServiceType)); + services.AddDbContext((_, option) => option.UseSqlServer(_connectionString)); + }); + } + } + } +} +``` + +### Understand the test structure + +#### Container lifecycle with IAsyncLifetime + +The outer `MsSqlTests` class implements `IAsyncLifetime`. xUnit calls +`InitializeAsync()` right after creating the class instance, which starts the +SQL Server container. After all tests complete, `DisposeAsync()` stops and +removes the container. + +```csharp +private readonly MsSqlContainer _msSqlContainer = new MsSqlBuilder().Build(); +``` + +`MsSqlBuilder().Build()` creates a pre-configured Microsoft SQL Server +container. Testcontainers modules follow best practices, so you don't need +to configure ports, passwords, or startup wait strategies yourself. + +#### Nested test class with IClassFixture + +The `IndexPageTests` class is nested inside `MsSqlTests` and implements +`IClassFixture`. This gives the test class access to the +container's private field and creates a clean hierarchy in the test explorer. + +#### Custom WebApplicationFactory + +Instead of using the SQLite-based factory, the nested +`CustomWebApplicationFactory` retrieves the connection string from the running +SQL Server container and passes it to `UseSqlServer()`: + +```csharp +private sealed class CustomWebApplicationFactory : WebApplicationFactory +{ + private readonly string _connectionString; + + public CustomWebApplicationFactory(MsSqlTests fixture) + { + _connectionString = fixture._msSqlContainer.GetConnectionString(); + } + + protected override void ConfigureWebHost(IWebHostBuilder builder) + { + builder.ConfigureServices(services => + { + services.Remove(services.SingleOrDefault(service => typeof(DbContextOptions) == service.ServiceType)); + services.Remove(services.SingleOrDefault(service => typeof(DbConnection) == service.ServiceType)); + services.AddDbContext((_, option) => option.UseSqlServer(_connectionString)); + }); + } +} +``` + +This factory: + +1. Removes the existing `DbContextOptions` registration +2. Removes the existing `DbConnection` registration +3. Adds a new `ApplicationDbContext` configured with the SQL Server connection + string from the Testcontainers-managed container + +> [!NOTE] +> The Microsoft SQL Server Docker image isn't compatible with ARM devices, such +> as Macs with Apple Silicon. You can use the +> [SqlEdge](https://www.nuget.org/packages/Testcontainers.SqlEdge) module or +> [Testcontainers Cloud](https://www.testcontainers.cloud/) as alternatives. + +## Run tests and next steps + +### Run the tests + +Run the tests from the solution root: + +```console +$ dotnet test ./RazorPagesProject.sln +``` + +The first run may take longer because Docker needs to pull the Microsoft SQL +Server image. On subsequent runs, the image is cached locally. + +You should see xUnit discover and run the tests, including the +`MsSqlTests.IndexPageTests` class. Testcontainers starts a SQL Server +container, the tests execute against it, and the container is stopped and +removed automatically after the tests finish. + +### Summary + +By replacing SQLite with a Testcontainers-managed Microsoft SQL Server +instance, the integration tests run against the same type of database used in +production. This approach catches database-specific issues early, such as +differences in SQL dialect, transaction behavior, or data type handling between +SQLite and SQL Server. + +The `MsSqlTests` class uses `IAsyncLifetime` to manage the container lifecycle, +and a nested `CustomWebApplicationFactory` wires the container's connection +string into the application's service configuration. You can apply this same +pattern to any database or service that Testcontainers supports. + +To learn more about Testcontainers, visit the +[Testcontainers overview](https://testcontainers.com/getting-started/). + +### Further reading + +- [Testcontainers for .NET documentation](https://dotnet.testcontainers.org/) +- [Testcontainers for .NET modules](https://dotnet.testcontainers.org/modules/) +- [Microsoft SQL Server module](https://www.nuget.org/packages/Testcontainers.MsSql) +- [Integration tests in ASP.NET Core](https://learn.microsoft.com/en-us/aspnet/core/test/integration-tests) diff --git a/content/guides/testcontainers-dotnet-aspnet-core/create-project.md b/content/guides/testcontainers-dotnet-aspnet-core/create-project.md deleted file mode 100644 index 9b1e2a93969c..000000000000 --- a/content/guides/testcontainers-dotnet-aspnet-core/create-project.md +++ /dev/null @@ -1,243 +0,0 @@ ---- -title: Set up the project -linkTitle: Create the project -description: Set up an ASP.NET Core Razor Pages project with integration test dependencies. -keywords: testcontainers, dotnet, asp.net core, getting started, project setup -weight: 10 ---- - -## Background - -This guide builds on top of Microsoft's -[Integration tests in ASP.NET Core](https://learn.microsoft.com/en-us/aspnet/core/test/integration-tests) -documentation. The original sample uses an in-memory SQLite database as the -backing store for integration tests. You'll replace SQLite with a real -Microsoft SQL Server instance running in a Docker container using -Testcontainers. - -You can find the original code sample in the -[dotnet/AspNetCore.Docs.Samples](https://github.com/dotnet/AspNetCore.Docs.Samples/tree/main/test/integration-tests/IntegrationTestsSample) -repository. - -## Clone the repository - -Clone the Testcontainers guide repository and change into the project -directory: - -```console -$ git clone https://github.com/testcontainers/tc-guide-testing-aspnet-core.git -$ cd tc-guide-testing-aspnet-core -``` - -## Project structure - -The solution contains two projects: - -```text -RazorPagesProject.sln -├── src/RazorPagesProject/ # ASP.NET Core Razor Pages app -└── tests/RazorPagesProject.Tests/ # xUnit integration tests -``` - -### Application project - -The application project (`src/RazorPagesProject/RazorPagesProject.csproj`) -is a Razor Pages web app that uses Entity Framework Core with SQLite as its -default database provider: - -```xml - - - - net9.0 - enable - - - - - - - - - all - runtime; build; native; contentfiles; analyzers; buildtransitive - - - - -``` - -The `ApplicationDbContext` stores `Message` entities and provides methods to -query and manage them: - -```csharp -public class ApplicationDbContext : IdentityDbContext -{ - public ApplicationDbContext(DbContextOptions options) - : base(options) - { - } - - public virtual DbSet Messages { get; set; } - - public async virtual Task> GetMessagesAsync() - { - return await Messages - .OrderBy(message => message.Text) - .AsNoTracking() - .ToListAsync(); - } - - public async virtual Task AddMessageAsync(Message message) - { - await Messages.AddAsync(message); - await SaveChangesAsync(); - } - - public async virtual Task DeleteAllMessagesAsync() - { - foreach (Message message in Messages) - { - Messages.Remove(message); - } - - await SaveChangesAsync(); - } - - public async virtual Task DeleteMessageAsync(int id) - { - var message = await Messages.FindAsync(id); - - if (message != null) - { - Messages.Remove(message); - await SaveChangesAsync(); - } - } - - public void Initialize() - { - Messages.AddRange(GetSeedingMessages()); - SaveChanges(); - } - - public static List GetSeedingMessages() - { - return new List() - { - new Message(){ Text = "You're standing on my scarf." }, - new Message(){ Text = "Would you like a jelly baby?" }, - new Message(){ Text = "To the rational mind, nothing is inexplicable; only unexplained." } - }; - } -} -``` - -### Test project - -The test project (`tests/RazorPagesProject.Tests/RazorPagesProject.Tests.csproj`) -includes xUnit, the ASP.NET Core testing infrastructure, and the -Testcontainers MSSQL module: - -```xml - - - - net9.0 - enable - - - - - - - - - - - - - all - runtime; build; native; contentfiles; analyzers; buildtransitive - - - - - - - - all - runtime; build; native; contentfiles; analyzers; buildtransitive - - - - - - - - - - Always - - - - -``` - -The key dependencies are: - -- `Microsoft.AspNetCore.Mvc.Testing` - provides `WebApplicationFactory` for - bootstrapping the app in tests -- `Microsoft.EntityFrameworkCore.SqlServer` - the SQL Server database provider - for Entity Framework Core -- `Testcontainers.MsSql` - the Testcontainers module for Microsoft SQL Server - -### Existing SQLite-based test factory - -The original project includes a `CustomWebApplicationFactory` that replaces -the application's database with an in-memory SQLite instance: - -```csharp -public class CustomWebApplicationFactory - : WebApplicationFactory where TProgram : class -{ - protected override void ConfigureWebHost(IWebHostBuilder builder) - { - builder.ConfigureServices(services => - { - var dbContextDescriptor = services.SingleOrDefault( - d => d.ServiceType == - typeof(DbContextOptions)); - - services.Remove(dbContextDescriptor); - - var dbConnectionDescriptor = services.SingleOrDefault( - d => d.ServiceType == - typeof(DbConnection)); - - services.Remove(dbConnectionDescriptor); - - // Create open SqliteConnection so EF won't automatically close it. - services.AddSingleton(container => - { - var connection = new SqliteConnection("DataSource=:memory:"); - connection.Open(); - - return connection; - }); - - services.AddDbContext((container, options) => - { - var connection = container.GetRequiredService(); - options.UseSqlite(connection); - }); - }); - - builder.UseEnvironment("Development"); - } -} -``` - -While this approach works, SQLite has behavioral differences from the database -you'd use in production. In the next section, you'll replace it with a -Testcontainers-managed Microsoft SQL Server instance. diff --git a/content/guides/testcontainers-dotnet-aspnet-core/run-tests.md b/content/guides/testcontainers-dotnet-aspnet-core/run-tests.md deleted file mode 100644 index affdc827232e..000000000000 --- a/content/guides/testcontainers-dotnet-aspnet-core/run-tests.md +++ /dev/null @@ -1,46 +0,0 @@ ---- -title: Run tests and next steps -linkTitle: Run tests -description: Run the Testcontainers-based integration tests and explore next steps. -keywords: testcontainers, dotnet, asp.net core, integration testing, run tests -weight: 30 ---- - -## Run the tests - -Run the tests from the solution root: - -```console -$ dotnet test ./RazorPagesProject.sln -``` - -The first run may take longer because Docker needs to pull the Microsoft SQL -Server image. On subsequent runs, the image is cached locally. - -You should see xUnit discover and run the tests, including the -`MsSqlTests.IndexPageTests` class. Testcontainers starts a SQL Server -container, the tests execute against it, and the container is stopped and -removed automatically after the tests finish. - -## Summary - -By replacing SQLite with a Testcontainers-managed Microsoft SQL Server -instance, the integration tests run against the same type of database used in -production. This approach catches database-specific issues early, such as -differences in SQL dialect, transaction behavior, or data type handling between -SQLite and SQL Server. - -The `MsSqlTests` class uses `IAsyncLifetime` to manage the container lifecycle, -and a nested `CustomWebApplicationFactory` wires the container's connection -string into the application's service configuration. You can apply this same -pattern to any database or service that Testcontainers supports. - -To learn more about Testcontainers, visit the -[Testcontainers overview](https://testcontainers.com/getting-started/). - -## Further reading - -- [Testcontainers for .NET documentation](https://dotnet.testcontainers.org/) -- [Testcontainers for .NET modules](https://dotnet.testcontainers.org/modules/) -- [Microsoft SQL Server module](https://www.nuget.org/packages/Testcontainers.MsSql) -- [Integration tests in ASP.NET Core](https://learn.microsoft.com/en-us/aspnet/core/test/integration-tests) diff --git a/content/guides/testcontainers-dotnet-aspnet-core/write-tests.md b/content/guides/testcontainers-dotnet-aspnet-core/write-tests.md deleted file mode 100644 index 7ce626f91dae..000000000000 --- a/content/guides/testcontainers-dotnet-aspnet-core/write-tests.md +++ /dev/null @@ -1,189 +0,0 @@ ---- -title: Write tests with Testcontainers -linkTitle: Write tests -description: Replace SQLite with a real Microsoft SQL Server using Testcontainers for .NET. -keywords: testcontainers, dotnet, asp.net core, sql server, integration testing, xunit -weight: 20 ---- - -The existing tests use an in-memory SQLite database. While convenient, this -doesn't match production behavior. You can replace it with a real Microsoft SQL -Server instance managed by Testcontainers. - -## Add dependencies - -Change to the test project directory and add the SQL Server Entity Framework -provider and the Testcontainers MSSQL module: - -```console -$ cd tests/RazorPagesProject.Tests -$ dotnet add package Microsoft.EntityFrameworkCore.SqlServer --version 7.0.0 -$ dotnet add package Testcontainers.MsSql --version 3.0.0 -``` - -> [!NOTE] -> Testcontainers for .NET offers a range of -> [modules](https://www.nuget.org/profiles/Testcontainers) that follow best -> practice configurations. - -## Create the test class - -Create a `MsSqlTests.cs` file in the `IntegrationTests` directory. This class -manages the SQL Server container lifecycle and contains a nested test class. - -```csharp -using System.Data.Common; -using System.Net; -using AngleSharp.Html.Dom; -using Microsoft.AspNetCore.Mvc.Testing; -using Microsoft.EntityFrameworkCore; -using RazorPagesProject.Data; -using RazorPagesProject.Tests.Helpers; -using Testcontainers.MsSql; -using Xunit; - -namespace RazorPagesProject.Tests.IntegrationTests; - -public sealed class MsSqlTests : IAsyncLifetime -{ - private readonly MsSqlContainer _msSqlContainer = new MsSqlBuilder().Build(); - - public Task InitializeAsync() - { - return _msSqlContainer.StartAsync(); - } - - public Task DisposeAsync() - { - return _msSqlContainer.DisposeAsync().AsTask(); - } - - public sealed class IndexPageTests : IClassFixture, IDisposable - { - private readonly WebApplicationFactory _webApplicationFactory; - - private readonly HttpClient _httpClient; - - public IndexPageTests(MsSqlTests fixture) - { - var clientOptions = new WebApplicationFactoryClientOptions(); - clientOptions.AllowAutoRedirect = false; - - _webApplicationFactory = new CustomWebApplicationFactory(fixture); - _httpClient = _webApplicationFactory.CreateClient(clientOptions); - } - - public void Dispose() - { - _webApplicationFactory.Dispose(); - } - - [Fact] - public async Task Post_DeleteAllMessagesHandler_ReturnsRedirectToRoot() - { - // Arrange - var defaultPage = await _httpClient.GetAsync("/") - .ConfigureAwait(false); - - var document = await HtmlHelpers.GetDocumentAsync(defaultPage) - .ConfigureAwait(false); - - // Act - var form = (IHtmlFormElement)document.QuerySelector("form[id='messages']"); - var submitButton = (IHtmlButtonElement)document.QuerySelector("button[id='deleteAllBtn']"); - - var response = await _httpClient.SendAsync(form, submitButton) - .ConfigureAwait(false); - - // Assert - Assert.Equal(HttpStatusCode.OK, defaultPage.StatusCode); - Assert.Equal(HttpStatusCode.Redirect, response.StatusCode); - Assert.Equal("/", response.Headers.Location.OriginalString); - } - - private sealed class CustomWebApplicationFactory : WebApplicationFactory - { - private readonly string _connectionString; - - public CustomWebApplicationFactory(MsSqlTests fixture) - { - _connectionString = fixture._msSqlContainer.GetConnectionString(); - } - - protected override void ConfigureWebHost(IWebHostBuilder builder) - { - builder.ConfigureServices(services => - { - services.Remove(services.SingleOrDefault(service => typeof(DbContextOptions) == service.ServiceType)); - services.Remove(services.SingleOrDefault(service => typeof(DbConnection) == service.ServiceType)); - services.AddDbContext((_, option) => option.UseSqlServer(_connectionString)); - }); - } - } - } -} -``` - -## Understand the test structure - -### Container lifecycle with IAsyncLifetime - -The outer `MsSqlTests` class implements `IAsyncLifetime`. xUnit calls -`InitializeAsync()` right after creating the class instance, which starts the -SQL Server container. After all tests complete, `DisposeAsync()` stops and -removes the container. - -```csharp -private readonly MsSqlContainer _msSqlContainer = new MsSqlBuilder().Build(); -``` - -`MsSqlBuilder().Build()` creates a pre-configured Microsoft SQL Server -container. Testcontainers modules follow best practices, so you don't need -to configure ports, passwords, or startup wait strategies yourself. - -### Nested test class with IClassFixture - -The `IndexPageTests` class is nested inside `MsSqlTests` and implements -`IClassFixture`. This gives the test class access to the -container's private field and creates a clean hierarchy in the test explorer. - -### Custom WebApplicationFactory - -Instead of using the SQLite-based factory, the nested -`CustomWebApplicationFactory` retrieves the connection string from the running -SQL Server container and passes it to `UseSqlServer()`: - -```csharp -private sealed class CustomWebApplicationFactory : WebApplicationFactory -{ - private readonly string _connectionString; - - public CustomWebApplicationFactory(MsSqlTests fixture) - { - _connectionString = fixture._msSqlContainer.GetConnectionString(); - } - - protected override void ConfigureWebHost(IWebHostBuilder builder) - { - builder.ConfigureServices(services => - { - services.Remove(services.SingleOrDefault(service => typeof(DbContextOptions) == service.ServiceType)); - services.Remove(services.SingleOrDefault(service => typeof(DbConnection) == service.ServiceType)); - services.AddDbContext((_, option) => option.UseSqlServer(_connectionString)); - }); - } -} -``` - -This factory: - -1. Removes the existing `DbContextOptions` registration -2. Removes the existing `DbConnection` registration -3. Adds a new `ApplicationDbContext` configured with the SQL Server connection - string from the Testcontainers-managed container - -> [!NOTE] -> The Microsoft SQL Server Docker image isn't compatible with ARM devices, such -> as Macs with Apple Silicon. You can use the -> [SqlEdge](https://www.nuget.org/packages/Testcontainers.SqlEdge) module or -> [Testcontainers Cloud](https://www.testcontainers.cloud/) as alternatives. diff --git a/content/guides/testcontainers-dotnet-getting-started/_index.md b/content/guides/testcontainers-dotnet-getting-started/_index.md index 6f784c941008..a6fde6a88d97 100644 --- a/content/guides/testcontainers-dotnet-getting-started/_index.md +++ b/content/guides/testcontainers-dotnet-getting-started/_index.md @@ -6,14 +6,16 @@ keywords: testcontainers, dotnet, csharp, testing, postgresql, integration testi summary: | Learn how to create a .NET application and test database interactions using Testcontainers for .NET with a real PostgreSQL instance. -toc_min: 1 -toc_max: 2 -tags: [testing-with-docker] -languages: [c-sharp] +aliases: + - /guides/testcontainers-dotnet-getting-started/create-project/ + - /guides/testcontainers-dotnet-getting-started/run-tests/ + - /guides/testcontainers-dotnet-getting-started/write-tests/ params: + tags: [testing] time: 20 minutes --- + In this guide, you will learn how to: @@ -32,3 +34,230 @@ In this guide, you will learn how to: > If you're new to Testcontainers, visit the > [Testcontainers overview](https://testcontainers.com/getting-started/) to learn more about > Testcontainers and the benefits of using it. + +## Create the .NET project + +### Set up the solution + +Create a .NET solution with source and test projects: + +```console +$ dotnet new sln -o TestcontainersDemo +$ cd TestcontainersDemo +$ dotnet new classlib -o CustomerService +$ dotnet sln add ./CustomerService/CustomerService.csproj +$ dotnet new xunit -o CustomerService.Tests +$ dotnet sln add ./CustomerService.Tests/CustomerService.Tests.csproj +$ dotnet add ./CustomerService.Tests/CustomerService.Tests.csproj reference ./CustomerService/CustomerService.csproj +``` + +Add the Npgsql dependency to the source project: + +```console +$ dotnet add ./CustomerService/CustomerService.csproj package Npgsql +``` + +### Implement the business logic + +Create a `Customer` record type: + +```csharp +namespace Customers; + +public readonly record struct Customer(long Id, string Name); +``` + +Create a `DbConnectionProvider` class to manage database connections: + +```csharp +using System.Data.Common; +using Npgsql; + +namespace Customers; + +public sealed class DbConnectionProvider +{ + private readonly string _connectionString; + + public DbConnectionProvider(string connectionString) + { + _connectionString = connectionString; + } + + public DbConnection GetConnection() + { + return new NpgsqlConnection(_connectionString); + } +} +``` + +Create the `CustomerService` class: + +```csharp +namespace Customers; + +public sealed class CustomerService +{ + private readonly DbConnectionProvider _dbConnectionProvider; + + public CustomerService(DbConnectionProvider dbConnectionProvider) + { + _dbConnectionProvider = dbConnectionProvider; + CreateCustomersTable(); + } + + public IEnumerable GetCustomers() + { + IList customers = new List(); + + using var connection = _dbConnectionProvider.GetConnection(); + using var command = connection.CreateCommand(); + command.CommandText = "SELECT id, name FROM customers"; + command.Connection?.Open(); + + using var dataReader = command.ExecuteReader(); + while (dataReader.Read()) + { + var id = dataReader.GetInt64(0); + var name = dataReader.GetString(1); + customers.Add(new Customer(id, name)); + } + + return customers; + } + + public void Create(Customer customer) + { + using var connection = _dbConnectionProvider.GetConnection(); + using var command = connection.CreateCommand(); + + var id = command.CreateParameter(); + id.ParameterName = "@id"; + id.Value = customer.Id; + + var name = command.CreateParameter(); + name.ParameterName = "@name"; + name.Value = customer.Name; + + command.CommandText = "INSERT INTO customers (id, name) VALUES(@id, @name)"; + command.Parameters.Add(id); + command.Parameters.Add(name); + command.Connection?.Open(); + command.ExecuteNonQuery(); + } + + private void CreateCustomersTable() + { + using var connection = _dbConnectionProvider.GetConnection(); + using var command = connection.CreateCommand(); + command.CommandText = "CREATE TABLE IF NOT EXISTS customers (id BIGINT NOT NULL, name VARCHAR NOT NULL, PRIMARY KEY (id))"; + command.Connection?.Open(); + command.ExecuteNonQuery(); + } +} +``` + +Here's what `CustomerService` does: + +- The constructor calls `CreateCustomersTable()` to ensure the table exists. +- `GetCustomers()` fetches all rows from the `customers` table and returns them as `Customer` objects. +- `Create()` inserts a customer record into the database. + +## Write tests with Testcontainers + +### Add Testcontainers dependencies + +Add the Testcontainers PostgreSQL module to the test project: + +```console +$ dotnet add ./CustomerService.Tests/CustomerService.Tests.csproj package Testcontainers.PostgreSql +``` + +### Write the test + +Create `CustomerServiceTest.cs` in the test project: + +```csharp +using Testcontainers.PostgreSql; + +namespace Customers.Tests; + +public sealed class CustomerServiceTest : IAsyncLifetime +{ + private readonly PostgreSqlContainer _postgres = new PostgreSqlBuilder() + .WithImage("postgres:16-alpine") + .Build(); + + public Task InitializeAsync() + { + return _postgres.StartAsync(); + } + + public Task DisposeAsync() + { + return _postgres.DisposeAsync().AsTask(); + } + + [Fact] + public void ShouldReturnTwoCustomers() + { + // Given + var customerService = new CustomerService(new DbConnectionProvider(_postgres.GetConnectionString())); + + // When + customerService.Create(new Customer(1, "George")); + customerService.Create(new Customer(2, "John")); + var customers = customerService.GetCustomers(); + + // Then + Assert.Equal(2, customers.Count()); + } +} +``` + +Here's what the test does: + +- Declares a `PostgreSqlContainer` using the `PostgreSqlBuilder` with the + `postgres:16-alpine` Docker image. +- Implements `IAsyncLifetime` for container lifecycle management: + - `InitializeAsync()` starts the container before the test runs. + - `DisposeAsync()` stops and removes the container after the test finishes. +- `ShouldReturnTwoCustomers()` creates a `CustomerService` with connection + details from the container, inserts two customers, fetches all customers, and + asserts the count. + +## Run tests and next steps + +### Run the tests + +Run the tests: + +```console +$ dotnet test +``` + +You can see in the output that Testcontainers pulls the Postgres Docker image +from Docker Hub (if not already available locally), starts the container, and +runs the test. + +Writing an integration test using Testcontainers works like writing a unit test +that you can run from your IDE. Your teammates can clone the project and run +tests without installing Postgres on their machines. + +### Summary + +The Testcontainers for .NET library helps you write integration tests using the +same type of database (Postgres) that you use in production, instead of mocks. +Because you aren't using mocks and instead talk to real services, you're free +to refactor code and still verify that the application works as expected. + +In addition to Postgres, Testcontainers provides dedicated +[modules](https://www.nuget.org/profiles/Testcontainers) for many SQL +databases, NoSQL databases, messaging queues, and more. + +To learn more about Testcontainers, visit the +[Testcontainers overview](https://testcontainers.com/getting-started/). + +### Further reading + +- [Testing an ASP.NET Core web app](https://testcontainers.com/guides/testing-an-aspnet-core-web-app/) diff --git a/content/guides/testcontainers-dotnet-getting-started/create-project.md b/content/guides/testcontainers-dotnet-getting-started/create-project.md deleted file mode 100644 index 18567d7d5b5a..000000000000 --- a/content/guides/testcontainers-dotnet-getting-started/create-project.md +++ /dev/null @@ -1,133 +0,0 @@ ---- -title: Create the .NET project -linkTitle: Create the project -description: Set up a .NET solution with a PostgreSQL-backed customer service. -keywords: testcontainers, dotnet, postgresql, getting started, project setup -weight: 10 ---- - -## Set up the solution - -Create a .NET solution with source and test projects: - -```console -$ dotnet new sln -o TestcontainersDemo -$ cd TestcontainersDemo -$ dotnet new classlib -o CustomerService -$ dotnet sln add ./CustomerService/CustomerService.csproj -$ dotnet new xunit -o CustomerService.Tests -$ dotnet sln add ./CustomerService.Tests/CustomerService.Tests.csproj -$ dotnet add ./CustomerService.Tests/CustomerService.Tests.csproj reference ./CustomerService/CustomerService.csproj -``` - -Add the Npgsql dependency to the source project: - -```console -$ dotnet add ./CustomerService/CustomerService.csproj package Npgsql -``` - -## Implement the business logic - -Create a `Customer` record type: - -```csharp -namespace Customers; - -public readonly record struct Customer(long Id, string Name); -``` - -Create a `DbConnectionProvider` class to manage database connections: - -```csharp -using System.Data.Common; -using Npgsql; - -namespace Customers; - -public sealed class DbConnectionProvider -{ - private readonly string _connectionString; - - public DbConnectionProvider(string connectionString) - { - _connectionString = connectionString; - } - - public DbConnection GetConnection() - { - return new NpgsqlConnection(_connectionString); - } -} -``` - -Create the `CustomerService` class: - -```csharp -namespace Customers; - -public sealed class CustomerService -{ - private readonly DbConnectionProvider _dbConnectionProvider; - - public CustomerService(DbConnectionProvider dbConnectionProvider) - { - _dbConnectionProvider = dbConnectionProvider; - CreateCustomersTable(); - } - - public IEnumerable GetCustomers() - { - IList customers = new List(); - - using var connection = _dbConnectionProvider.GetConnection(); - using var command = connection.CreateCommand(); - command.CommandText = "SELECT id, name FROM customers"; - command.Connection?.Open(); - - using var dataReader = command.ExecuteReader(); - while (dataReader.Read()) - { - var id = dataReader.GetInt64(0); - var name = dataReader.GetString(1); - customers.Add(new Customer(id, name)); - } - - return customers; - } - - public void Create(Customer customer) - { - using var connection = _dbConnectionProvider.GetConnection(); - using var command = connection.CreateCommand(); - - var id = command.CreateParameter(); - id.ParameterName = "@id"; - id.Value = customer.Id; - - var name = command.CreateParameter(); - name.ParameterName = "@name"; - name.Value = customer.Name; - - command.CommandText = "INSERT INTO customers (id, name) VALUES(@id, @name)"; - command.Parameters.Add(id); - command.Parameters.Add(name); - command.Connection?.Open(); - command.ExecuteNonQuery(); - } - - private void CreateCustomersTable() - { - using var connection = _dbConnectionProvider.GetConnection(); - using var command = connection.CreateCommand(); - command.CommandText = "CREATE TABLE IF NOT EXISTS customers (id BIGINT NOT NULL, name VARCHAR NOT NULL, PRIMARY KEY (id))"; - command.Connection?.Open(); - command.ExecuteNonQuery(); - } -} -``` - -Here's what `CustomerService` does: - -- The constructor calls `CreateCustomersTable()` to ensure the table exists. -- `GetCustomers()` fetches all rows from the `customers` table and returns them as `Customer` objects. -- `Create()` inserts a customer record into the database. diff --git a/content/guides/testcontainers-dotnet-getting-started/run-tests.md b/content/guides/testcontainers-dotnet-getting-started/run-tests.md deleted file mode 100644 index bcaa66452784..000000000000 --- a/content/guides/testcontainers-dotnet-getting-started/run-tests.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -title: Run tests and next steps -linkTitle: Run tests -description: Run your Testcontainers-based integration tests and explore next steps. -keywords: testcontainers, dotnet, postgresql, integration testing, run tests -weight: 30 ---- - -## Run the tests - -Run the tests: - -```console -$ dotnet test -``` - -You can see in the output that Testcontainers pulls the Postgres Docker image -from Docker Hub (if not already available locally), starts the container, and -runs the test. - -Writing an integration test using Testcontainers works like writing a unit test -that you can run from your IDE. Your teammates can clone the project and run -tests without installing Postgres on their machines. - -## Summary - -The Testcontainers for .NET library helps you write integration tests using the -same type of database (Postgres) that you use in production, instead of mocks. -Because you aren't using mocks and instead talk to real services, you're free -to refactor code and still verify that the application works as expected. - -In addition to Postgres, Testcontainers provides dedicated -[modules](https://www.nuget.org/profiles/Testcontainers) for many SQL -databases, NoSQL databases, messaging queues, and more. - -To learn more about Testcontainers, visit the -[Testcontainers overview](https://testcontainers.com/getting-started/). - -## Further reading - -- [Testing an ASP.NET Core web app](https://testcontainers.com/guides/testing-an-aspnet-core-web-app/) diff --git a/content/guides/testcontainers-dotnet-getting-started/write-tests.md b/content/guides/testcontainers-dotnet-getting-started/write-tests.md deleted file mode 100644 index ca3528496693..000000000000 --- a/content/guides/testcontainers-dotnet-getting-started/write-tests.md +++ /dev/null @@ -1,68 +0,0 @@ ---- -title: Write tests with Testcontainers -linkTitle: Write tests -description: Write integration tests using Testcontainers for .NET and xUnit with a real PostgreSQL database. -keywords: testcontainers, dotnet, postgresql, xunit, integration testing -weight: 20 ---- - -## Add Testcontainers dependencies - -Add the Testcontainers PostgreSQL module to the test project: - -```console -$ dotnet add ./CustomerService.Tests/CustomerService.Tests.csproj package Testcontainers.PostgreSql -``` - -## Write the test - -Create `CustomerServiceTest.cs` in the test project: - -```csharp -using Testcontainers.PostgreSql; - -namespace Customers.Tests; - -public sealed class CustomerServiceTest : IAsyncLifetime -{ - private readonly PostgreSqlContainer _postgres = new PostgreSqlBuilder() - .WithImage("postgres:16-alpine") - .Build(); - - public Task InitializeAsync() - { - return _postgres.StartAsync(); - } - - public Task DisposeAsync() - { - return _postgres.DisposeAsync().AsTask(); - } - - [Fact] - public void ShouldReturnTwoCustomers() - { - // Given - var customerService = new CustomerService(new DbConnectionProvider(_postgres.GetConnectionString())); - - // When - customerService.Create(new Customer(1, "George")); - customerService.Create(new Customer(2, "John")); - var customers = customerService.GetCustomers(); - - // Then - Assert.Equal(2, customers.Count()); - } -} -``` - -Here's what the test does: - -- Declares a `PostgreSqlContainer` using the `PostgreSqlBuilder` with the - `postgres:16-alpine` Docker image. -- Implements `IAsyncLifetime` for container lifecycle management: - - `InitializeAsync()` starts the container before the test runs. - - `DisposeAsync()` stops and removes the container after the test finishes. -- `ShouldReturnTwoCustomers()` creates a `CustomerService` with connection - details from the container, inserts two customers, fetches all customers, and - asserts the count. diff --git a/content/guides/testcontainers-go-getting-started/_index.md b/content/guides/testcontainers-go-getting-started/_index.md index 5ec598979d08..650d9fb1aef9 100644 --- a/content/guides/testcontainers-go-getting-started/_index.md +++ b/content/guides/testcontainers-go-getting-started/_index.md @@ -6,14 +6,17 @@ keywords: testcontainers, go, golang, testing, postgresql, integration testing summary: | Learn how to create a Go application and test database interactions using Testcontainers for Go with a real PostgreSQL instance. -toc_min: 1 -toc_max: 2 -tags: [testing-with-docker] -languages: [go] +aliases: + - /guides/testcontainers-go-getting-started/create-project/ + - /guides/testcontainers-go-getting-started/run-tests/ + - /guides/testcontainers-go-getting-started/test-suites/ + - /guides/testcontainers-go-getting-started/write-tests/ params: + tags: [testing] time: 20 minutes --- + In this guide, you will learn how to: @@ -34,3 +37,377 @@ In this guide, you will learn how to: > If you're new to Testcontainers, visit the > [Testcontainers overview](https://testcontainers.com/getting-started/) to learn more about > Testcontainers and the benefits of using it. + +## Create the Go project + +### Initialize the project + +Start by creating a Go project. + +```console +$ mkdir testcontainers-go-demo +$ cd testcontainers-go-demo +$ go mod init github.com/testcontainers/testcontainers-go-demo +``` + +This guide uses the [jackc/pgx](https://github.com/jackc/pgx) PostgreSQL +driver to interact with the Postgres database and the testcontainers-go +[Postgres module](https://golang.testcontainers.org/modules/postgres/) to +spin up a Postgres Docker instance for testing. It also uses +[testify](https://github.com/stretchr/testify) for running multiple tests +as a suite and for writing assertions. + +Install these dependencies: + +```console +$ go get github.com/jackc/pgx/v5 +$ go get github.com/testcontainers/testcontainers-go +$ go get github.com/testcontainers/testcontainers-go/modules/postgres +$ go get github.com/stretchr/testify +``` + +### Create Customer struct + +Create a `types.go` file in the `customer` package and define the `Customer` +struct to model the customer details: + +```go +package customer + +type Customer struct { + Id int + Name string + Email string +} +``` + +### Create Repository + +Next, create `customer/repo.go`, define the `Repository` struct, and add +methods to create a customer and get a customer by email: + +```go +package customer + +import ( + "context" + "fmt" + "os" + + "github.com/jackc/pgx/v5" +) + +type Repository struct { + conn *pgx.Conn +} + +func NewRepository(ctx context.Context, connStr string) (*Repository, error) { + conn, err := pgx.Connect(ctx, connStr) + if err != nil { + _, _ = fmt.Fprintf(os.Stderr, "Unable to connect to database: %v\n", err) + return nil, err + } + return &Repository{ + conn: conn, + }, nil +} + +func (r Repository) CreateCustomer(ctx context.Context, customer Customer) (Customer, error) { + err := r.conn.QueryRow(ctx, + "INSERT INTO customers (name, email) VALUES ($1, $2) RETURNING id", + customer.Name, customer.Email).Scan(&customer.Id) + return customer, err +} + +func (r Repository) GetCustomerByEmail(ctx context.Context, email string) (Customer, error) { + var customer Customer + query := "SELECT id, name, email FROM customers WHERE email = $1" + err := r.conn.QueryRow(ctx, query, email). + Scan(&customer.Id, &customer.Name, &customer.Email) + if err != nil { + return Customer{}, err + } + return customer, nil +} +``` + +Here's what the code does: + +- `Repository` holds a `*pgx.Conn` for performing database operations. +- `NewRepository(connStr)` takes a database connection string and initializes a `Repository`. +- `CreateCustomer()` and `GetCustomerByEmail()` are methods on the `Repository` receiver that insert and query customer records. + +## Write tests with Testcontainers + +You have the `Repository` implementation ready, but for testing you need a +PostgreSQL database. You can use testcontainers-go to spin up a Postgres +database in a Docker container and run your tests against that database. + +### Set up the test database + +In real applications you might use a database migration tool, but for this +guide, use a script to initialize the database. + +Create a `testdata/init-db.sql` file to create the `CUSTOMERS` table and +insert sample data: + +```sql +CREATE TABLE IF NOT EXISTS customers (id serial, name varchar(255), email varchar(255)); + +INSERT INTO customers(name, email) VALUES ('John', 'john@gmail.com'); +``` + +### Understand the testcontainers-go API + +The testcontainers-go library provides the generic `Container` abstraction +that can run any containerized service. To further simplify, testcontainers-go +provides technology-specific modules that reduce boilerplate and provide a +functional options pattern to construct the container instance. + +For example, `PostgresContainer` provides `WithDatabase()`, +`WithUsername()`, `WithPassword()`, and other functions to set various +properties of Postgres containers. + +### Write the test + +Create the `customer/repo_test.go` file and implement the test: + +```go +package customer + +import ( + "context" + "path/filepath" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "github.com/testcontainers/testcontainers-go" + "github.com/testcontainers/testcontainers-go/modules/postgres" +) + +func TestCustomerRepository(t *testing.T) { + ctx := context.Background() + + ctr, err := postgres.Run(ctx, + "postgres:16-alpine", + postgres.WithInitScripts(filepath.Join("..", "testdata", "init-db.sql")), + postgres.WithDatabase("test-db"), + postgres.WithUsername("postgres"), + postgres.WithPassword("postgres"), + postgres.BasicWaitStrategies(), + ) + testcontainers.CleanupContainer(t, ctr) + require.NoError(t, err) + + connStr, err := ctr.ConnectionString(ctx, "sslmode=disable") + require.NoError(t, err) + + customerRepo, err := NewRepository(ctx, connStr) + require.NoError(t, err) + + c, err := customerRepo.CreateCustomer(ctx, Customer{ + Name: "Henry", + Email: "henry@gmail.com", + }) + assert.NoError(t, err) + assert.NotNil(t, c) + + customer, err := customerRepo.GetCustomerByEmail(ctx, "henry@gmail.com") + assert.NoError(t, err) + assert.NotNil(t, customer) + assert.Equal(t, "Henry", customer.Name) + assert.Equal(t, "henry@gmail.com", customer.Email) +} +``` + +Here's what the test does: + +- Calls `postgres.Run()` with the `postgres:16-alpine` Docker image as the + first argument. This is the v0.41.0 API — the image is a required positional + parameter instead of an option. +- Configures initialization scripts using `WithInitScripts(...)` so that the + `CUSTOMERS` table is created and sample data is inserted after the database + starts. +- Uses `postgres.BasicWaitStrategies()` which combines waiting for the Postgres + log message and for the port to be ready. This replaces manual wait strategy + configuration. +- Calls `testcontainers.CleanupContainer(t, ctr)` right after `postgres.Run()`. + This registers automatic cleanup with the test framework, replacing the manual + `t.Cleanup` and `Terminate` pattern. +- Obtains the database `ConnectionString` from the container and initializes a + `Repository`. +- Creates a customer with the email `henry@gmail.com` and verifies that the + customer exists in the database. + +## Reuse containers with test suites + +In the previous section, you saw how to spin up a Postgres Docker container +for a single test. But often you have multiple tests in a single file, and you +may want to reuse the same Postgres Docker container for all of them. + +You can use the [testify suite](https://pkg.go.dev/github.com/stretchr/testify/suite) +package to implement common test setup and teardown actions. + +### Extract container setup + +First, extract the `PostgresContainer` creation logic into a separate file +called `testhelpers/containers.go`: + +```go +package testhelpers + +import ( + "context" + "path/filepath" + "testing" + + "github.com/stretchr/testify/require" + "github.com/testcontainers/testcontainers-go" + "github.com/testcontainers/testcontainers-go/modules/postgres" +) + +type PostgresContainer struct { + *postgres.PostgresContainer + ConnectionString string +} + +func CreatePostgresContainer(t *testing.T, ctx context.Context) *PostgresContainer { + t.Helper() + + ctr, err := postgres.Run(ctx, + "postgres:16-alpine", + postgres.WithInitScripts(filepath.Join("..", "testdata", "init-db.sql")), + postgres.WithDatabase("test-db"), + postgres.WithUsername("postgres"), + postgres.WithPassword("postgres"), + postgres.BasicWaitStrategies(), + ) + testcontainers.CleanupContainer(t, ctr) + require.NoError(t, err) + + connStr, err := ctr.ConnectionString(ctx, "sslmode=disable") + require.NoError(t, err) + + return &PostgresContainer{ + PostgresContainer: ctr, + ConnectionString: connStr, + } +} +``` + +In `containers.go`, `PostgresContainer` extends the testcontainers-go +`PostgresContainer` to provide easy access to `ConnectionString`. The +`CreatePostgresContainer()` function accepts `*testing.T` as its first +parameter, calls `t.Helper()` so that test failures point to the caller, +and uses `testcontainers.CleanupContainer()` to register automatic cleanup. + +### Write the test suite + +Create `customer/repo_suite_test.go` and implement tests for creating +a customer and getting a customer by email using the testify suite package: + +```go +package customer + +import ( + "context" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "github.com/stretchr/testify/suite" + "github.com/testcontainers/testcontainers-go-demo/testhelpers" +) + +type CustomerRepoTestSuite struct { + suite.Suite + pgContainer *testhelpers.PostgresContainer + repository *Repository + ctx context.Context +} + +func (suite *CustomerRepoTestSuite) SetupSuite() { + suite.ctx = context.Background() + suite.pgContainer = testhelpers.CreatePostgresContainer(suite.T(), suite.ctx) + + repository, err := NewRepository(suite.ctx, suite.pgContainer.ConnectionString) + require.NoError(suite.T(), err) + suite.repository = repository +} + +func (suite *CustomerRepoTestSuite) TestCreateCustomer() { + t := suite.T() + + customer, err := suite.repository.CreateCustomer(suite.ctx, Customer{ + Name: "Henry", + Email: "henry@gmail.com", + }) + require.NoError(t, err) + assert.NotNil(t, customer.Id) +} + +func (suite *CustomerRepoTestSuite) TestGetCustomerByEmail() { + t := suite.T() + + customer, err := suite.repository.GetCustomerByEmail(suite.ctx, "john@gmail.com") + require.NoError(t, err) + assert.Equal(t, "John", customer.Name) + assert.Equal(t, "john@gmail.com", customer.Email) +} + +func TestCustomerRepoTestSuite(t *testing.T) { + suite.Run(t, new(CustomerRepoTestSuite)) +} +``` + +Here's what the code does: + +- `CustomerRepoTestSuite` extends `suite.Suite` and includes fields shared + across multiple tests. +- `SetupSuite()` runs once before all tests. It calls + `CreatePostgresContainer(suite.T(), ...)` which handles cleanup registration + automatically via `CleanupContainer`, so no `TearDownSuite()` is needed. +- `TestCreateCustomer()` uses `require.NoError()` for the create operation + (fail immediately if it errors) and `assert.NotNil()` for the ID check. +- `TestGetCustomerByEmail()` uses `require.NoError()` then asserts on the + returned values. +- `TestCustomerRepoTestSuite(t *testing.T)` runs the test suite when you + execute `go test`. + +> [!TIP] +> For the purpose of this guide, the tests don't reset data in the database. +> In practice, it's a good idea to reset the database to a known state before +> running each test. + +## Run tests and next steps + +### Run the tests + +Run all the tests using `go test ./...`. Optionally add the `-v` flag for +verbose output: + +```console +$ go test -v ./... +``` + +You should see two Postgres Docker containers start automatically: one for the +suite and its two tests, and another for the initial standalone test. All tests +should pass. After the tests finish, the containers are stopped and removed +automatically. + +### Summary + +The Testcontainers for Go library helps you write integration tests by using +the same type of database (Postgres) that you use in production, instead of +mocks. Because you aren't using mocks and instead talk to real services, you're +free to refactor code and still verify that the application works as expected. + +To learn more about Testcontainers, visit the +[Testcontainers overview](https://testcontainers.com/getting-started/). + +### Further reading + +- [Testcontainers for Go documentation](https://golang.testcontainers.org/) +- [Testcontainers for Go quickstart](https://golang.testcontainers.org/quickstart/) +- [Testcontainers Postgres module for Go](https://golang.testcontainers.org/modules/postgres/) diff --git a/content/guides/testcontainers-go-getting-started/create-project.md b/content/guides/testcontainers-go-getting-started/create-project.md deleted file mode 100644 index ce18f4a016f4..000000000000 --- a/content/guides/testcontainers-go-getting-started/create-project.md +++ /dev/null @@ -1,104 +0,0 @@ ---- -title: Create the Go project -linkTitle: Create the project -description: Set up a Go project with a PostgreSQL-backed repository. -keywords: testcontainers, go, golang, postgresql, getting started, project setup -weight: 10 ---- - -## Initialize the project - -Start by creating a Go project. - -```console -$ mkdir testcontainers-go-demo -$ cd testcontainers-go-demo -$ go mod init github.com/testcontainers/testcontainers-go-demo -``` - -This guide uses the [jackc/pgx](https://github.com/jackc/pgx) PostgreSQL -driver to interact with the Postgres database and the testcontainers-go -[Postgres module](https://golang.testcontainers.org/modules/postgres/) to -spin up a Postgres Docker instance for testing. It also uses -[testify](https://github.com/stretchr/testify) for running multiple tests -as a suite and for writing assertions. - -Install these dependencies: - -```console -$ go get github.com/jackc/pgx/v5 -$ go get github.com/testcontainers/testcontainers-go -$ go get github.com/testcontainers/testcontainers-go/modules/postgres -$ go get github.com/stretchr/testify -``` - -## Create Customer struct - -Create a `types.go` file in the `customer` package and define the `Customer` -struct to model the customer details: - -```go -package customer - -type Customer struct { - Id int - Name string - Email string -} -``` - -## Create Repository - -Next, create `customer/repo.go`, define the `Repository` struct, and add -methods to create a customer and get a customer by email: - -```go -package customer - -import ( - "context" - "fmt" - "os" - - "github.com/jackc/pgx/v5" -) - -type Repository struct { - conn *pgx.Conn -} - -func NewRepository(ctx context.Context, connStr string) (*Repository, error) { - conn, err := pgx.Connect(ctx, connStr) - if err != nil { - _, _ = fmt.Fprintf(os.Stderr, "Unable to connect to database: %v\n", err) - return nil, err - } - return &Repository{ - conn: conn, - }, nil -} - -func (r Repository) CreateCustomer(ctx context.Context, customer Customer) (Customer, error) { - err := r.conn.QueryRow(ctx, - "INSERT INTO customers (name, email) VALUES ($1, $2) RETURNING id", - customer.Name, customer.Email).Scan(&customer.Id) - return customer, err -} - -func (r Repository) GetCustomerByEmail(ctx context.Context, email string) (Customer, error) { - var customer Customer - query := "SELECT id, name, email FROM customers WHERE email = $1" - err := r.conn.QueryRow(ctx, query, email). - Scan(&customer.Id, &customer.Name, &customer.Email) - if err != nil { - return Customer{}, err - } - return customer, nil -} -``` - -Here's what the code does: - -- `Repository` holds a `*pgx.Conn` for performing database operations. -- `NewRepository(connStr)` takes a database connection string and initializes a `Repository`. -- `CreateCustomer()` and `GetCustomerByEmail()` are methods on the `Repository` receiver that insert and query customer records. diff --git a/content/guides/testcontainers-go-getting-started/run-tests.md b/content/guides/testcontainers-go-getting-started/run-tests.md deleted file mode 100644 index 194902dd04d0..000000000000 --- a/content/guides/testcontainers-go-getting-started/run-tests.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -title: Run tests and next steps -linkTitle: Run tests -description: Run your Testcontainers-based integration tests and explore next steps. -keywords: testcontainers, go, golang, integration testing, run tests -weight: 40 ---- - -## Run the tests - -Run all the tests using `go test ./...`. Optionally add the `-v` flag for -verbose output: - -```console -$ go test -v ./... -``` - -You should see two Postgres Docker containers start automatically: one for the -suite and its two tests, and another for the initial standalone test. All tests -should pass. After the tests finish, the containers are stopped and removed -automatically. - -## Summary - -The Testcontainers for Go library helps you write integration tests by using -the same type of database (Postgres) that you use in production, instead of -mocks. Because you aren't using mocks and instead talk to real services, you're -free to refactor code and still verify that the application works as expected. - -To learn more about Testcontainers, visit the -[Testcontainers overview](https://testcontainers.com/getting-started/). - -## Further reading - -- [Testcontainers for Go documentation](https://golang.testcontainers.org/) -- [Testcontainers for Go quickstart](https://golang.testcontainers.org/quickstart/) -- [Testcontainers Postgres module for Go](https://golang.testcontainers.org/modules/postgres/) diff --git a/content/guides/testcontainers-go-getting-started/test-suites.md b/content/guides/testcontainers-go-getting-started/test-suites.md deleted file mode 100644 index 51739335a8fd..000000000000 --- a/content/guides/testcontainers-go-getting-started/test-suites.md +++ /dev/null @@ -1,145 +0,0 @@ ---- -title: Reuse containers with test suites -linkTitle: Test suites -description: Share a single Postgres container across multiple tests using testify suites. -keywords: testcontainers, go, golang, testify, test suites, container reuse -weight: 30 ---- - -In the previous section, you saw how to spin up a Postgres Docker container -for a single test. But often you have multiple tests in a single file, and you -may want to reuse the same Postgres Docker container for all of them. - -You can use the [testify suite](https://pkg.go.dev/github.com/stretchr/testify/suite) -package to implement common test setup and teardown actions. - -## Extract container setup - -First, extract the `PostgresContainer` creation logic into a separate file -called `testhelpers/containers.go`: - -```go -package testhelpers - -import ( - "context" - "path/filepath" - "testing" - - "github.com/stretchr/testify/require" - "github.com/testcontainers/testcontainers-go" - "github.com/testcontainers/testcontainers-go/modules/postgres" -) - -type PostgresContainer struct { - *postgres.PostgresContainer - ConnectionString string -} - -func CreatePostgresContainer(t *testing.T, ctx context.Context) *PostgresContainer { - t.Helper() - - ctr, err := postgres.Run(ctx, - "postgres:16-alpine", - postgres.WithInitScripts(filepath.Join("..", "testdata", "init-db.sql")), - postgres.WithDatabase("test-db"), - postgres.WithUsername("postgres"), - postgres.WithPassword("postgres"), - postgres.BasicWaitStrategies(), - ) - testcontainers.CleanupContainer(t, ctr) - require.NoError(t, err) - - connStr, err := ctr.ConnectionString(ctx, "sslmode=disable") - require.NoError(t, err) - - return &PostgresContainer{ - PostgresContainer: ctr, - ConnectionString: connStr, - } -} -``` - -In `containers.go`, `PostgresContainer` extends the testcontainers-go -`PostgresContainer` to provide easy access to `ConnectionString`. The -`CreatePostgresContainer()` function accepts `*testing.T` as its first -parameter, calls `t.Helper()` so that test failures point to the caller, -and uses `testcontainers.CleanupContainer()` to register automatic cleanup. - -## Write the test suite - -Create `customer/repo_suite_test.go` and implement tests for creating -a customer and getting a customer by email using the testify suite package: - -```go -package customer - -import ( - "context" - "testing" - - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" - "github.com/stretchr/testify/suite" - "github.com/testcontainers/testcontainers-go-demo/testhelpers" -) - -type CustomerRepoTestSuite struct { - suite.Suite - pgContainer *testhelpers.PostgresContainer - repository *Repository - ctx context.Context -} - -func (suite *CustomerRepoTestSuite) SetupSuite() { - suite.ctx = context.Background() - suite.pgContainer = testhelpers.CreatePostgresContainer(suite.T(), suite.ctx) - - repository, err := NewRepository(suite.ctx, suite.pgContainer.ConnectionString) - require.NoError(suite.T(), err) - suite.repository = repository -} - -func (suite *CustomerRepoTestSuite) TestCreateCustomer() { - t := suite.T() - - customer, err := suite.repository.CreateCustomer(suite.ctx, Customer{ - Name: "Henry", - Email: "henry@gmail.com", - }) - require.NoError(t, err) - assert.NotNil(t, customer.Id) -} - -func (suite *CustomerRepoTestSuite) TestGetCustomerByEmail() { - t := suite.T() - - customer, err := suite.repository.GetCustomerByEmail(suite.ctx, "john@gmail.com") - require.NoError(t, err) - assert.Equal(t, "John", customer.Name) - assert.Equal(t, "john@gmail.com", customer.Email) -} - -func TestCustomerRepoTestSuite(t *testing.T) { - suite.Run(t, new(CustomerRepoTestSuite)) -} -``` - -Here's what the code does: - -- `CustomerRepoTestSuite` extends `suite.Suite` and includes fields shared - across multiple tests. -- `SetupSuite()` runs once before all tests. It calls - `CreatePostgresContainer(suite.T(), ...)` which handles cleanup registration - automatically via `CleanupContainer`, so no `TearDownSuite()` is needed. -- `TestCreateCustomer()` uses `require.NoError()` for the create operation - (fail immediately if it errors) and `assert.NotNil()` for the ID check. -- `TestGetCustomerByEmail()` uses `require.NoError()` then asserts on the - returned values. -- `TestCustomerRepoTestSuite(t *testing.T)` runs the test suite when you - execute `go test`. - -> [!TIP] -> For the purpose of this guide, the tests don't reset data in the database. -> In practice, it's a good idea to reset the database to a known state before -> running each test. diff --git a/content/guides/testcontainers-go-getting-started/write-tests.md b/content/guides/testcontainers-go-getting-started/write-tests.md deleted file mode 100644 index 0fbc936bf175..000000000000 --- a/content/guides/testcontainers-go-getting-started/write-tests.md +++ /dev/null @@ -1,108 +0,0 @@ ---- -title: Write tests with Testcontainers -linkTitle: Write tests -description: Write your first integration test using testcontainers-go and PostgreSQL. -keywords: testcontainers, go, golang, postgresql, integration testing -weight: 20 ---- - -You have the `Repository` implementation ready, but for testing you need a -PostgreSQL database. You can use testcontainers-go to spin up a Postgres -database in a Docker container and run your tests against that database. - -## Set up the test database - -In real applications you might use a database migration tool, but for this -guide, use a script to initialize the database. - -Create a `testdata/init-db.sql` file to create the `CUSTOMERS` table and -insert sample data: - -```sql -CREATE TABLE IF NOT EXISTS customers (id serial, name varchar(255), email varchar(255)); - -INSERT INTO customers(name, email) VALUES ('John', 'john@gmail.com'); -``` - -## Understand the testcontainers-go API - -The testcontainers-go library provides the generic `Container` abstraction -that can run any containerized service. To further simplify, testcontainers-go -provides technology-specific modules that reduce boilerplate and provide a -functional options pattern to construct the container instance. - -For example, `PostgresContainer` provides `WithDatabase()`, -`WithUsername()`, `WithPassword()`, and other functions to set various -properties of Postgres containers. - -## Write the test - -Create the `customer/repo_test.go` file and implement the test: - -```go -package customer - -import ( - "context" - "path/filepath" - "testing" - - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" - "github.com/testcontainers/testcontainers-go" - "github.com/testcontainers/testcontainers-go/modules/postgres" -) - -func TestCustomerRepository(t *testing.T) { - ctx := context.Background() - - ctr, err := postgres.Run(ctx, - "postgres:16-alpine", - postgres.WithInitScripts(filepath.Join("..", "testdata", "init-db.sql")), - postgres.WithDatabase("test-db"), - postgres.WithUsername("postgres"), - postgres.WithPassword("postgres"), - postgres.BasicWaitStrategies(), - ) - testcontainers.CleanupContainer(t, ctr) - require.NoError(t, err) - - connStr, err := ctr.ConnectionString(ctx, "sslmode=disable") - require.NoError(t, err) - - customerRepo, err := NewRepository(ctx, connStr) - require.NoError(t, err) - - c, err := customerRepo.CreateCustomer(ctx, Customer{ - Name: "Henry", - Email: "henry@gmail.com", - }) - assert.NoError(t, err) - assert.NotNil(t, c) - - customer, err := customerRepo.GetCustomerByEmail(ctx, "henry@gmail.com") - assert.NoError(t, err) - assert.NotNil(t, customer) - assert.Equal(t, "Henry", customer.Name) - assert.Equal(t, "henry@gmail.com", customer.Email) -} -``` - -Here's what the test does: - -- Calls `postgres.Run()` with the `postgres:16-alpine` Docker image as the - first argument. This is the v0.41.0 API — the image is a required positional - parameter instead of an option. -- Configures initialization scripts using `WithInitScripts(...)` so that the - `CUSTOMERS` table is created and sample data is inserted after the database - starts. -- Uses `postgres.BasicWaitStrategies()` which combines waiting for the Postgres - log message and for the port to be ready. This replaces manual wait strategy - configuration. -- Calls `testcontainers.CleanupContainer(t, ctr)` right after `postgres.Run()`. - This registers automatic cleanup with the test framework, replacing the manual - `t.Cleanup` and `Terminate` pattern. -- Obtains the database `ConnectionString` from the container and initializes a - `Repository`. -- Creates a customer with the email `henry@gmail.com` and verifies that the - customer exists in the database. diff --git a/content/guides/testcontainers-java-aws-localstack/_index.md b/content/guides/testcontainers-java-aws-localstack/_index.md index 85978649e34a..fa6295b5ee42 100644 --- a/content/guides/testcontainers-java-aws-localstack/_index.md +++ b/content/guides/testcontainers-java-aws-localstack/_index.md @@ -6,14 +6,16 @@ keywords: testcontainers, java, spring boot, testing, aws, localstack, s3, sqs, summary: | Learn how to create a Spring Boot application with Spring Cloud AWS, then test S3 and SQS integrations using Testcontainers and LocalStack. -toc_min: 1 -toc_max: 2 -tags: [testing-with-docker] -languages: [java] +aliases: + - /guides/testcontainers-java-aws-localstack/create-project/ + - /guides/testcontainers-java-aws-localstack/run-tests/ + - /guides/testcontainers-java-aws-localstack/write-tests/ params: + tags: [testing] time: 25 minutes --- + In this guide, you will learn how to: @@ -32,3 +34,417 @@ In this guide, you will learn how to: > If you're new to Testcontainers, visit the > [Testcontainers overview](https://testcontainers.com/getting-started/) to learn more about > Testcontainers and the benefits of using it. + +## Create the Spring Boot project + +### Set up the project + +Create a Spring Boot project from [Spring Initializr](https://start.spring.io) +by selecting the **Testcontainers** starter. Spring Cloud AWS starters are not +available on Spring Initializr, so you need to add them manually. + +Alternatively, clone the +[guide repository](https://github.com/testcontainers/tc-guide-testing-aws-service-integrations-using-localstack). + +Add the Spring Cloud AWS BOM to your dependency management and add the S3, SQS +starters as dependencies. Testcontainers provides a +[LocalStack module](https://testcontainers.com/modules/localstack/) for testing +AWS service integrations. You also need +[Awaitility](http://www.awaitility.org/) for testing asynchronous SQS +processing. + +The key dependencies in `pom.xml` are: + +```xml + + 17 + 2.0.4 + 3.0.3 + + + + + org.springframework.boot + spring-boot-starter-web + + + io.awspring.cloud + spring-cloud-aws-starter-s3 + + + io.awspring.cloud + spring-cloud-aws-starter-sqs + + + org.springframework.boot + spring-boot-starter-test + test + + + org.springframework.boot + spring-boot-testcontainers + test + + + org.testcontainers + testcontainers-junit-jupiter + test + + + org.testcontainers + testcontainers-localstack + test + + + org.awaitility + awaitility + test + + + + + + + io.awspring.cloud + spring-cloud-aws-dependencies + ${awspring.version} + pom + import + + + +``` + +### Create the configuration properties + +To make the SQS queue and S3 bucket names configurable, create an +`ApplicationProperties` record: + +```java +package com.testcontainers.demo; + +import org.springframework.boot.context.properties.ConfigurationProperties; + +@ConfigurationProperties(prefix = "app") +public record ApplicationProperties(String queue, String bucket) {} +``` + +Then add `@ConfigurationPropertiesScan` to the main application class so that +Spring automatically scans for `@ConfigurationProperties`-annotated classes and +registers them as beans: + +```java +package com.testcontainers.demo; + +import org.springframework.boot.SpringApplication; +import org.springframework.boot.autoconfigure.SpringBootApplication; +import org.springframework.boot.context.properties.ConfigurationPropertiesScan; + +@SpringBootApplication +@ConfigurationPropertiesScan +public class Application { + + public static void main(String[] args) { + SpringApplication.run(Application.class, args); + } +} +``` + +### Implement StorageService for S3 + +Spring Cloud AWS provides higher-level abstractions like `S3Template` with +convenience methods for uploading and downloading files. Create a +`StorageService` class: + +```java +package com.testcontainers.demo; + +import io.awspring.cloud.s3.S3Template; +import java.io.IOException; +import java.io.InputStream; +import org.springframework.stereotype.Service; + +@Service +public class StorageService { + + private final S3Template s3Template; + + public StorageService(S3Template s3Template) { + this.s3Template = s3Template; + } + + public void upload(String bucketName, String key, InputStream stream) { + this.s3Template.upload(bucketName, key, stream); + } + + public InputStream download(String bucketName, String key) + throws IOException { + return this.s3Template.download(bucketName, key).getInputStream(); + } + + public String downloadAsString(String bucketName, String key) + throws IOException { + try (InputStream is = this.download(bucketName, key)) { + return new String(is.readAllBytes()); + } + } +} +``` + +### Create the SQS message model + +Create a `Message` record that represents the payload you send to the SQS +queue: + +```java +package com.testcontainers.demo; + +import java.util.UUID; + +public record Message(UUID uuid, String content) {} +``` + +### Implement the message sender + +Create `MessageSender`, which uses `SqsTemplate` to publish messages: + +```java +package com.testcontainers.demo; + +import io.awspring.cloud.sqs.operations.SqsTemplate; +import org.springframework.stereotype.Service; + +@Service +public class MessageSender { + + private final SqsTemplate sqsTemplate; + + public MessageSender(SqsTemplate sqsTemplate) { + this.sqsTemplate = sqsTemplate; + } + + public void publish(String queueName, Message message) { + sqsTemplate.send(to -> to.queue(queueName).payload(message)); + } +} +``` + +### Implement the message listener + +Create `MessageListener` with a handler method annotated with `@SqsListener`. +When a message arrives, the listener uploads the content to an S3 bucket using +the message UUID as the key: + +```java +package com.testcontainers.demo; + +import io.awspring.cloud.sqs.annotation.SqsListener; +import java.io.ByteArrayInputStream; +import java.nio.charset.StandardCharsets; +import org.springframework.stereotype.Service; + +@Service +public class MessageListener { + + private final StorageService storageService; + private final ApplicationProperties properties; + + public MessageListener( + StorageService storageService, + ApplicationProperties properties + ) { + this.storageService = storageService; + this.properties = properties; + } + + @SqsListener(queueNames = { "${app.queue}" }) + public void handle(Message message) { + String bucketName = this.properties.bucket(); + String key = message.uuid().toString(); + ByteArrayInputStream is = new ByteArrayInputStream( + message.content().getBytes(StandardCharsets.UTF_8) + ); + this.storageService.upload(bucketName, key, is); + } +} +``` + +The `${app.queue}` expression reads the queue name from application +configuration instead of hard-coding it. + +## Write tests with Testcontainers + +To test the application, you need a running LocalStack instance that emulates +the AWS S3 and SQS services. Testcontainers spins up LocalStack in a Docker +container and `@DynamicPropertySource` connects it to Spring Cloud AWS. + +### Configure the test container + +You can start a LocalStack container and configure the Spring Cloud AWS +properties to talk to it instead of actual AWS services. The properties you +need to set are: + +```properties +spring.cloud.aws.s3.endpoint=http://localhost:4566 +spring.cloud.aws.sqs.endpoint=http://localhost:4566 +spring.cloud.aws.credentials.access-key=noop +spring.cloud.aws.credentials.secret-key=noop +spring.cloud.aws.region.static=us-east-1 +``` + +For testing, use an ephemeral container that starts on a random available port +so that you can run multiple builds in CI in parallel without port conflicts. + +### Write the test + +Create `MessageListenerTest.java`: + +```java +package com.testcontainers.demo; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.awaitility.Awaitility.await; +import static org.testcontainers.containers.localstack.LocalStackContainer.Service.S3; +import static org.testcontainers.containers.localstack.LocalStackContainer.Service.SQS; + +import java.io.IOException; +import java.time.Duration; +import java.util.UUID; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.test.context.DynamicPropertyRegistry; +import org.springframework.test.context.DynamicPropertySource; +import org.testcontainers.containers.localstack.LocalStackContainer; +import org.testcontainers.junit.jupiter.Container; +import org.testcontainers.junit.jupiter.Testcontainers; +import org.testcontainers.utility.DockerImageName; + +@SpringBootTest +@Testcontainers +class MessageListenerTest { + + @Container + static LocalStackContainer localStack = new LocalStackContainer( + DockerImageName.parse("localstack/localstack:3.0") + ); + + static final String BUCKET_NAME = UUID.randomUUID().toString(); + static final String QUEUE_NAME = UUID.randomUUID().toString(); + + @DynamicPropertySource + static void overrideProperties(DynamicPropertyRegistry registry) { + registry.add("app.bucket", () -> BUCKET_NAME); + registry.add("app.queue", () -> QUEUE_NAME); + registry.add( + "spring.cloud.aws.region.static", + () -> localStack.getRegion() + ); + registry.add( + "spring.cloud.aws.credentials.access-key", + () -> localStack.getAccessKey() + ); + registry.add( + "spring.cloud.aws.credentials.secret-key", + () -> localStack.getSecretKey() + ); + registry.add( + "spring.cloud.aws.s3.endpoint", + () -> localStack.getEndpointOverride(S3).toString() + ); + registry.add( + "spring.cloud.aws.sqs.endpoint", + () -> localStack.getEndpointOverride(SQS).toString() + ); + } + + @BeforeAll + static void beforeAll() throws IOException, InterruptedException { + localStack.execInContainer("awslocal", "s3", "mb", "s3://" + BUCKET_NAME); + localStack.execInContainer( + "awslocal", + "sqs", + "create-queue", + "--queue-name", + QUEUE_NAME + ); + } + + @Autowired + StorageService storageService; + + @Autowired + MessageSender publisher; + + @Autowired + ApplicationProperties properties; + + @Test + void shouldHandleMessageSuccessfully() { + Message message = new Message(UUID.randomUUID(), "Hello World"); + publisher.publish(properties.queue(), message); + + await() + .pollInterval(Duration.ofSeconds(2)) + .atMost(Duration.ofSeconds(10)) + .ignoreExceptions() + .untilAsserted(() -> { + String msg = storageService.downloadAsString( + properties.bucket(), + message.uuid().toString() + ); + assertThat(msg).isEqualTo("Hello World"); + }); + } +} +``` + +Here's what the test does: + +- `@SpringBootTest` starts the full Spring application context. +- The Testcontainers JUnit 5 annotations `@Testcontainers` and `@Container` + manage the lifecycle of a `LocalStackContainer` instance. +- `@DynamicPropertySource` obtains the dynamic S3 and SQS endpoint URLs, + region, access key, and secret key from the container, and registers them as + Spring Cloud AWS configuration properties. +- `@BeforeAll` creates the required SQS queue and S3 bucket using the + `awslocal` CLI tool that comes pre-installed in the LocalStack Docker image. + The `localStack.execInContainer()` API runs commands inside the container. +- `shouldHandleMessageSuccessfully()` publishes a `Message` to the SQS queue. + The listener receives the message and stores its content in the S3 bucket + with the UUID as the key. Awaitility waits up to 10 seconds for the expected + content to appear in the bucket. + +## Run tests and next steps + +### Run the tests + +```console +$ ./mvnw test +``` + +Or with Gradle: + +```console +$ ./gradlew test +``` + +You should see the LocalStack Docker container start and the test pass. After +the tests finish, the container stops and is removed automatically. + +### Summary + +LocalStack lets you develop and test AWS-based applications locally. +The Testcontainers LocalStack module makes it straightforward to write +integration tests by using ephemeral LocalStack containers that start on random +ports with no external setup required. + +To learn more about Testcontainers, visit the +[Testcontainers overview](https://testcontainers.com/getting-started/). + +### Further reading + +- [Testcontainers LocalStack module](https://java.testcontainers.org/modules/localstack/) +- [Getting started with Testcontainers for Java](https://java.testcontainers.org/quickstart/junit_5_quickstart/) +- [Spring Cloud AWS documentation](https://docs.awspring.io/spring-cloud-aws/docs/3.0.3/reference/html/index.html) diff --git a/content/guides/testcontainers-java-aws-localstack/create-project.md b/content/guides/testcontainers-java-aws-localstack/create-project.md deleted file mode 100644 index 1bc2a577ceaf..000000000000 --- a/content/guides/testcontainers-java-aws-localstack/create-project.md +++ /dev/null @@ -1,242 +0,0 @@ ---- -title: Create the Spring Boot project -linkTitle: Create the project -description: Set up a Spring Boot project with Spring Cloud AWS, S3, and SQS. -keywords: testcontainers, java, spring boot, aws, localstack, s3, sqs, project setup -weight: 10 ---- - -## Set up the project - -Create a Spring Boot project from [Spring Initializr](https://start.spring.io) -by selecting the **Testcontainers** starter. Spring Cloud AWS starters are not -available on Spring Initializr, so you need to add them manually. - -Alternatively, clone the -[guide repository](https://github.com/testcontainers/tc-guide-testing-aws-service-integrations-using-localstack). - -Add the Spring Cloud AWS BOM to your dependency management and add the S3, SQS -starters as dependencies. Testcontainers provides a -[LocalStack module](https://testcontainers.com/modules/localstack/) for testing -AWS service integrations. You also need -[Awaitility](http://www.awaitility.org/) for testing asynchronous SQS -processing. - -The key dependencies in `pom.xml` are: - -```xml - - 17 - 2.0.4 - 3.0.3 - - - - - org.springframework.boot - spring-boot-starter-web - - - io.awspring.cloud - spring-cloud-aws-starter-s3 - - - io.awspring.cloud - spring-cloud-aws-starter-sqs - - - org.springframework.boot - spring-boot-starter-test - test - - - org.springframework.boot - spring-boot-testcontainers - test - - - org.testcontainers - testcontainers-junit-jupiter - test - - - org.testcontainers - testcontainers-localstack - test - - - org.awaitility - awaitility - test - - - - - - - io.awspring.cloud - spring-cloud-aws-dependencies - ${awspring.version} - pom - import - - - -``` - -## Create the configuration properties - -To make the SQS queue and S3 bucket names configurable, create an -`ApplicationProperties` record: - -```java -package com.testcontainers.demo; - -import org.springframework.boot.context.properties.ConfigurationProperties; - -@ConfigurationProperties(prefix = "app") -public record ApplicationProperties(String queue, String bucket) {} -``` - -Then add `@ConfigurationPropertiesScan` to the main application class so that -Spring automatically scans for `@ConfigurationProperties`-annotated classes and -registers them as beans: - -```java -package com.testcontainers.demo; - -import org.springframework.boot.SpringApplication; -import org.springframework.boot.autoconfigure.SpringBootApplication; -import org.springframework.boot.context.properties.ConfigurationPropertiesScan; - -@SpringBootApplication -@ConfigurationPropertiesScan -public class Application { - - public static void main(String[] args) { - SpringApplication.run(Application.class, args); - } -} -``` - -## Implement StorageService for S3 - -Spring Cloud AWS provides higher-level abstractions like `S3Template` with -convenience methods for uploading and downloading files. Create a -`StorageService` class: - -```java -package com.testcontainers.demo; - -import io.awspring.cloud.s3.S3Template; -import java.io.IOException; -import java.io.InputStream; -import org.springframework.stereotype.Service; - -@Service -public class StorageService { - - private final S3Template s3Template; - - public StorageService(S3Template s3Template) { - this.s3Template = s3Template; - } - - public void upload(String bucketName, String key, InputStream stream) { - this.s3Template.upload(bucketName, key, stream); - } - - public InputStream download(String bucketName, String key) - throws IOException { - return this.s3Template.download(bucketName, key).getInputStream(); - } - - public String downloadAsString(String bucketName, String key) - throws IOException { - try (InputStream is = this.download(bucketName, key)) { - return new String(is.readAllBytes()); - } - } -} -``` - -## Create the SQS message model - -Create a `Message` record that represents the payload you send to the SQS -queue: - -```java -package com.testcontainers.demo; - -import java.util.UUID; - -public record Message(UUID uuid, String content) {} -``` - -## Implement the message sender - -Create `MessageSender`, which uses `SqsTemplate` to publish messages: - -```java -package com.testcontainers.demo; - -import io.awspring.cloud.sqs.operations.SqsTemplate; -import org.springframework.stereotype.Service; - -@Service -public class MessageSender { - - private final SqsTemplate sqsTemplate; - - public MessageSender(SqsTemplate sqsTemplate) { - this.sqsTemplate = sqsTemplate; - } - - public void publish(String queueName, Message message) { - sqsTemplate.send(to -> to.queue(queueName).payload(message)); - } -} -``` - -## Implement the message listener - -Create `MessageListener` with a handler method annotated with `@SqsListener`. -When a message arrives, the listener uploads the content to an S3 bucket using -the message UUID as the key: - -```java -package com.testcontainers.demo; - -import io.awspring.cloud.sqs.annotation.SqsListener; -import java.io.ByteArrayInputStream; -import java.nio.charset.StandardCharsets; -import org.springframework.stereotype.Service; - -@Service -public class MessageListener { - - private final StorageService storageService; - private final ApplicationProperties properties; - - public MessageListener( - StorageService storageService, - ApplicationProperties properties - ) { - this.storageService = storageService; - this.properties = properties; - } - - @SqsListener(queueNames = { "${app.queue}" }) - public void handle(Message message) { - String bucketName = this.properties.bucket(); - String key = message.uuid().toString(); - ByteArrayInputStream is = new ByteArrayInputStream( - message.content().getBytes(StandardCharsets.UTF_8) - ); - this.storageService.upload(bucketName, key, is); - } -} -``` - -The `${app.queue}` expression reads the queue name from application -configuration instead of hard-coding it. diff --git a/content/guides/testcontainers-java-aws-localstack/run-tests.md b/content/guides/testcontainers-java-aws-localstack/run-tests.md deleted file mode 100644 index e3521ed5f2e0..000000000000 --- a/content/guides/testcontainers-java-aws-localstack/run-tests.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -title: Run tests and next steps -linkTitle: Run tests -description: Run your Testcontainers-based Spring Cloud AWS integration tests and explore next steps. -keywords: testcontainers, java, spring boot, aws, localstack, integration testing -weight: 30 ---- - -## Run the tests - -```console -$ ./mvnw test -``` - -Or with Gradle: - -```console -$ ./gradlew test -``` - -You should see the LocalStack Docker container start and the test pass. After -the tests finish, the container stops and is removed automatically. - -## Summary - -LocalStack lets you develop and test AWS-based applications locally. -The Testcontainers LocalStack module makes it straightforward to write -integration tests by using ephemeral LocalStack containers that start on random -ports with no external setup required. - -To learn more about Testcontainers, visit the -[Testcontainers overview](https://testcontainers.com/getting-started/). - -## Further reading - -- [Testcontainers LocalStack module](https://java.testcontainers.org/modules/localstack/) -- [Getting started with Testcontainers for Java](https://java.testcontainers.org/quickstart/junit_5_quickstart/) -- [Spring Cloud AWS documentation](https://docs.awspring.io/spring-cloud-aws/docs/3.0.3/reference/html/index.html) diff --git a/content/guides/testcontainers-java-aws-localstack/write-tests.md b/content/guides/testcontainers-java-aws-localstack/write-tests.md deleted file mode 100644 index caf7b49d1b7b..000000000000 --- a/content/guides/testcontainers-java-aws-localstack/write-tests.md +++ /dev/null @@ -1,149 +0,0 @@ ---- -title: Write tests with Testcontainers -linkTitle: Write tests -description: Test Spring Cloud AWS S3 and SQS integration using Testcontainers and LocalStack. -keywords: testcontainers, java, spring boot, aws, localstack, s3, sqs, integration testing -weight: 20 ---- - -To test the application, you need a running LocalStack instance that emulates -the AWS S3 and SQS services. Testcontainers spins up LocalStack in a Docker -container and `@DynamicPropertySource` connects it to Spring Cloud AWS. - -## Configure the test container - -You can start a LocalStack container and configure the Spring Cloud AWS -properties to talk to it instead of actual AWS services. The properties you -need to set are: - -```properties -spring.cloud.aws.s3.endpoint=http://localhost:4566 -spring.cloud.aws.sqs.endpoint=http://localhost:4566 -spring.cloud.aws.credentials.access-key=noop -spring.cloud.aws.credentials.secret-key=noop -spring.cloud.aws.region.static=us-east-1 -``` - -For testing, use an ephemeral container that starts on a random available port -so that you can run multiple builds in CI in parallel without port conflicts. - -## Write the test - -Create `MessageListenerTest.java`: - -```java -package com.testcontainers.demo; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.awaitility.Awaitility.await; -import static org.testcontainers.containers.localstack.LocalStackContainer.Service.S3; -import static org.testcontainers.containers.localstack.LocalStackContainer.Service.SQS; - -import java.io.IOException; -import java.time.Duration; -import java.util.UUID; -import org.junit.jupiter.api.BeforeAll; -import org.junit.jupiter.api.Test; -import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.test.context.DynamicPropertyRegistry; -import org.springframework.test.context.DynamicPropertySource; -import org.testcontainers.containers.localstack.LocalStackContainer; -import org.testcontainers.junit.jupiter.Container; -import org.testcontainers.junit.jupiter.Testcontainers; -import org.testcontainers.utility.DockerImageName; - -@SpringBootTest -@Testcontainers -class MessageListenerTest { - - @Container - static LocalStackContainer localStack = new LocalStackContainer( - DockerImageName.parse("localstack/localstack:3.0") - ); - - static final String BUCKET_NAME = UUID.randomUUID().toString(); - static final String QUEUE_NAME = UUID.randomUUID().toString(); - - @DynamicPropertySource - static void overrideProperties(DynamicPropertyRegistry registry) { - registry.add("app.bucket", () -> BUCKET_NAME); - registry.add("app.queue", () -> QUEUE_NAME); - registry.add( - "spring.cloud.aws.region.static", - () -> localStack.getRegion() - ); - registry.add( - "spring.cloud.aws.credentials.access-key", - () -> localStack.getAccessKey() - ); - registry.add( - "spring.cloud.aws.credentials.secret-key", - () -> localStack.getSecretKey() - ); - registry.add( - "spring.cloud.aws.s3.endpoint", - () -> localStack.getEndpointOverride(S3).toString() - ); - registry.add( - "spring.cloud.aws.sqs.endpoint", - () -> localStack.getEndpointOverride(SQS).toString() - ); - } - - @BeforeAll - static void beforeAll() throws IOException, InterruptedException { - localStack.execInContainer("awslocal", "s3", "mb", "s3://" + BUCKET_NAME); - localStack.execInContainer( - "awslocal", - "sqs", - "create-queue", - "--queue-name", - QUEUE_NAME - ); - } - - @Autowired - StorageService storageService; - - @Autowired - MessageSender publisher; - - @Autowired - ApplicationProperties properties; - - @Test - void shouldHandleMessageSuccessfully() { - Message message = new Message(UUID.randomUUID(), "Hello World"); - publisher.publish(properties.queue(), message); - - await() - .pollInterval(Duration.ofSeconds(2)) - .atMost(Duration.ofSeconds(10)) - .ignoreExceptions() - .untilAsserted(() -> { - String msg = storageService.downloadAsString( - properties.bucket(), - message.uuid().toString() - ); - assertThat(msg).isEqualTo("Hello World"); - }); - } -} -``` - -Here's what the test does: - -- `@SpringBootTest` starts the full Spring application context. -- The Testcontainers JUnit 5 annotations `@Testcontainers` and `@Container` - manage the lifecycle of a `LocalStackContainer` instance. -- `@DynamicPropertySource` obtains the dynamic S3 and SQS endpoint URLs, - region, access key, and secret key from the container, and registers them as - Spring Cloud AWS configuration properties. -- `@BeforeAll` creates the required SQS queue and S3 bucket using the - `awslocal` CLI tool that comes pre-installed in the LocalStack Docker image. - The `localStack.execInContainer()` API runs commands inside the container. -- `shouldHandleMessageSuccessfully()` publishes a `Message` to the SQS queue. - The listener receives the message and stores its content in the S3 bucket - with the UUID as the key. Awaitility waits up to 10 seconds for the expected - content to appear in the bucket. diff --git a/content/guides/testcontainers-java-getting-started/_index.md b/content/guides/testcontainers-java-getting-started/_index.md index 1a4ee3b837fa..0e8750894002 100644 --- a/content/guides/testcontainers-java-getting-started/_index.md +++ b/content/guides/testcontainers-java-getting-started/_index.md @@ -6,14 +6,16 @@ keywords: testcontainers, java, testing, postgresql, integration testing, junit, summary: | Learn how to create a Java application and test database interactions using Testcontainers for Java with a real PostgreSQL instance. -toc_min: 1 -toc_max: 2 -tags: [testing-with-docker] -languages: [java] +aliases: + - /guides/testcontainers-java-getting-started/create-project/ + - /guides/testcontainers-java-getting-started/run-tests/ + - /guides/testcontainers-java-getting-started/write-tests/ params: + tags: [testing] time: 20 minutes --- + In this guide, you will learn how to: @@ -33,3 +35,292 @@ In this guide, you will learn how to: > If you're new to Testcontainers, visit the > [Testcontainers overview](https://testcontainers.com/getting-started/) to learn more about > Testcontainers and the benefits of using it. + +## Create the Java project + +### Set up the Maven project + +Create a Java project with Maven from your preferred IDE. This guide uses +Maven, but you can use Gradle if you prefer. Add the following dependencies +to `pom.xml`: + +```xml + + + org.postgresql + postgresql + 42.7.3 + + + ch.qos.logback + logback-classic + 1.5.6 + + + org.junit.jupiter + junit-jupiter + 5.10.2 + test + + + + + + + org.apache.maven.plugins + maven-surefire-plugin + 3.2.5 + + + +``` + +This adds the Postgres JDBC driver, logback for logging, JUnit 5 for testing, +and the latest `maven-surefire-plugin` for JUnit 5 support. + +### Implement the business logic + +Create a `Customer` record: + +```java +package com.testcontainers.demo; + +public record Customer(Long id, String name) {} +``` + +Create a `DBConnectionProvider` class to hold JDBC connection parameters and +provide a database `Connection`: + +```java +package com.testcontainers.demo; + +import java.sql.Connection; +import java.sql.DriverManager; + +class DBConnectionProvider { + + private final String url; + private final String username; + private final String password; + + public DBConnectionProvider(String url, String username, String password) { + this.url = url; + this.username = username; + this.password = password; + } + + Connection getConnection() { + try { + return DriverManager.getConnection(url, username, password); + } catch (Exception e) { + throw new RuntimeException(e); + } + } +} +``` + +Create the `CustomerService` class: + +```java +package com.testcontainers.demo; + +import java.sql.Connection; +import java.sql.PreparedStatement; +import java.sql.ResultSet; +import java.sql.SQLException; +import java.util.ArrayList; +import java.util.List; + +public class CustomerService { + + private final DBConnectionProvider connectionProvider; + + public CustomerService(DBConnectionProvider connectionProvider) { + this.connectionProvider = connectionProvider; + createCustomersTableIfNotExists(); + } + + public void createCustomer(Customer customer) { + try (Connection conn = this.connectionProvider.getConnection()) { + PreparedStatement pstmt = conn.prepareStatement( + "insert into customers(id,name) values(?,?)" + ); + pstmt.setLong(1, customer.id()); + pstmt.setString(2, customer.name()); + pstmt.execute(); + } catch (SQLException e) { + throw new RuntimeException(e); + } + } + + public List getAllCustomers() { + List customers = new ArrayList<>(); + + try (Connection conn = this.connectionProvider.getConnection()) { + PreparedStatement pstmt = conn.prepareStatement( + "select id,name from customers" + ); + ResultSet rs = pstmt.executeQuery(); + while (rs.next()) { + long id = rs.getLong("id"); + String name = rs.getString("name"); + customers.add(new Customer(id, name)); + } + } catch (SQLException e) { + throw new RuntimeException(e); + } + return customers; + } + + private void createCustomersTableIfNotExists() { + try (Connection conn = this.connectionProvider.getConnection()) { + PreparedStatement pstmt = conn.prepareStatement( + """ + create table if not exists customers ( + id bigint not null, + name varchar not null, + primary key (id) + ) + """ + ); + pstmt.execute(); + } catch (SQLException e) { + throw new RuntimeException(e); + } + } +} +``` + +Here's what `CustomerService` does: + +- The constructor calls `createCustomersTableIfNotExists()` to ensure the table exists. +- `createCustomer()` inserts a customer record into the database. +- `getAllCustomers()` fetches all rows from the `customers` table and returns a list of `Customer` objects. + +## Write tests with Testcontainers + +You have the `CustomerService` implementation ready, but for testing you need a +PostgreSQL database. You can use Testcontainers to spin up a Postgres database +in a Docker container and run your tests against it. + +### Add Testcontainers dependencies + +Add the Testcontainers PostgreSQL module as a test dependency in `pom.xml`: + +```xml + + org.testcontainers + testcontainers-postgresql + 2.0.4 + test + +``` + +Since the application uses a Postgres database, the Testcontainers Postgres +module provides a `PostgreSQLContainer` class for managing the container. + +### Write the test + +Create `CustomerServiceTest.java` under `src/test/java`: + +```java +package com.testcontainers.demo; + +import static org.junit.jupiter.api.Assertions.assertEquals; + +import java.util.List; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.testcontainers.postgresql.PostgreSQLContainer; + +class CustomerServiceTest { + + static PostgreSQLContainer postgres = new PostgreSQLContainer( + "postgres:16-alpine" + ); + + CustomerService customerService; + + @BeforeAll + static void beforeAll() { + postgres.start(); + } + + @AfterAll + static void afterAll() { + postgres.stop(); + } + + @BeforeEach + void setUp() { + DBConnectionProvider connectionProvider = new DBConnectionProvider( + postgres.getJdbcUrl(), + postgres.getUsername(), + postgres.getPassword() + ); + customerService = new CustomerService(connectionProvider); + } + + @Test + void shouldGetCustomers() { + customerService.createCustomer(new Customer(1L, "George")); + customerService.createCustomer(new Customer(2L, "John")); + + List customers = customerService.getAllCustomers(); + assertEquals(2, customers.size()); + } +} +``` + +Here's what the test does: + +- Declares a `PostgreSQLContainer` with the `postgres:16-alpine` Docker image. +- The `@BeforeAll` callback starts the Postgres container before any test + methods run. +- The `@BeforeEach` callback creates a `DBConnectionProvider` using the JDBC + connection parameters from the container, then creates a `CustomerService`. + The `CustomerService` constructor creates the `customers` table if it + doesn't exist. +- `shouldGetCustomers()` inserts 2 customer records, fetches all customers, + and asserts the count. +- The `@AfterAll` callback stops the container after all test methods finish. + +## Run tests and next steps + +### Run the tests + +Run the tests using Maven: + +```console +$ mvn test +``` + +You can see in the logs that Testcontainers pulls the Postgres Docker image +from Docker Hub (if not already available locally), starts the container, and +runs the test. + +Writing an integration test using Testcontainers works like writing a unit test +that you can run from your IDE. Your teammates can clone the project +and run tests without installing Postgres on their machines. + +### Summary + +The Testcontainers for Java library helps you write integration tests using the +same type of database (Postgres) that you use in production, instead of mocks. +Because you aren't using mocks and instead talk to real services, you're free +to refactor code and still verify that the application works as expected. + +In addition to Postgres, Testcontainers provides dedicated modules for many +SQL databases, NoSQL databases, messaging queues, and more. You can use +Testcontainers to run any containerized dependency for your tests. + +To learn more about Testcontainers, visit the +[Testcontainers overview](https://testcontainers.com/getting-started/). + +### Further reading + +- [Testcontainers container lifecycle management using JUnit 5](https://testcontainers.com/guides/testcontainers-container-lifecycle/) +- [Replace H2 with a real database for testing](https://testcontainers.com/guides/replace-h2-with-real-database-for-testing/) +- [Getting started with Testcontainers in a Java Spring Boot project](https://testcontainers.com/guides/testing-spring-boot-rest-api-using-testcontainers/) diff --git a/content/guides/testcontainers-java-getting-started/create-project.md b/content/guides/testcontainers-java-getting-started/create-project.md deleted file mode 100644 index 19ef95958c31..000000000000 --- a/content/guides/testcontainers-java-getting-started/create-project.md +++ /dev/null @@ -1,166 +0,0 @@ ---- -title: Create the Java project -linkTitle: Create the project -description: Set up a Java project with Maven and implement a PostgreSQL-backed customer service. -keywords: testcontainers, java, maven, postgresql, getting started, project setup -weight: 10 ---- - -## Set up the Maven project - -Create a Java project with Maven from your preferred IDE. This guide uses -Maven, but you can use Gradle if you prefer. Add the following dependencies -to `pom.xml`: - -```xml - - - org.postgresql - postgresql - 42.7.3 - - - ch.qos.logback - logback-classic - 1.5.6 - - - org.junit.jupiter - junit-jupiter - 5.10.2 - test - - - - - - - org.apache.maven.plugins - maven-surefire-plugin - 3.2.5 - - - -``` - -This adds the Postgres JDBC driver, logback for logging, JUnit 5 for testing, -and the latest `maven-surefire-plugin` for JUnit 5 support. - -## Implement the business logic - -Create a `Customer` record: - -```java -package com.testcontainers.demo; - -public record Customer(Long id, String name) {} -``` - -Create a `DBConnectionProvider` class to hold JDBC connection parameters and -provide a database `Connection`: - -```java -package com.testcontainers.demo; - -import java.sql.Connection; -import java.sql.DriverManager; - -class DBConnectionProvider { - - private final String url; - private final String username; - private final String password; - - public DBConnectionProvider(String url, String username, String password) { - this.url = url; - this.username = username; - this.password = password; - } - - Connection getConnection() { - try { - return DriverManager.getConnection(url, username, password); - } catch (Exception e) { - throw new RuntimeException(e); - } - } -} -``` - -Create the `CustomerService` class: - -```java -package com.testcontainers.demo; - -import java.sql.Connection; -import java.sql.PreparedStatement; -import java.sql.ResultSet; -import java.sql.SQLException; -import java.util.ArrayList; -import java.util.List; - -public class CustomerService { - - private final DBConnectionProvider connectionProvider; - - public CustomerService(DBConnectionProvider connectionProvider) { - this.connectionProvider = connectionProvider; - createCustomersTableIfNotExists(); - } - - public void createCustomer(Customer customer) { - try (Connection conn = this.connectionProvider.getConnection()) { - PreparedStatement pstmt = conn.prepareStatement( - "insert into customers(id,name) values(?,?)" - ); - pstmt.setLong(1, customer.id()); - pstmt.setString(2, customer.name()); - pstmt.execute(); - } catch (SQLException e) { - throw new RuntimeException(e); - } - } - - public List getAllCustomers() { - List customers = new ArrayList<>(); - - try (Connection conn = this.connectionProvider.getConnection()) { - PreparedStatement pstmt = conn.prepareStatement( - "select id,name from customers" - ); - ResultSet rs = pstmt.executeQuery(); - while (rs.next()) { - long id = rs.getLong("id"); - String name = rs.getString("name"); - customers.add(new Customer(id, name)); - } - } catch (SQLException e) { - throw new RuntimeException(e); - } - return customers; - } - - private void createCustomersTableIfNotExists() { - try (Connection conn = this.connectionProvider.getConnection()) { - PreparedStatement pstmt = conn.prepareStatement( - """ - create table if not exists customers ( - id bigint not null, - name varchar not null, - primary key (id) - ) - """ - ); - pstmt.execute(); - } catch (SQLException e) { - throw new RuntimeException(e); - } - } -} -``` - -Here's what `CustomerService` does: - -- The constructor calls `createCustomersTableIfNotExists()` to ensure the table exists. -- `createCustomer()` inserts a customer record into the database. -- `getAllCustomers()` fetches all rows from the `customers` table and returns a list of `Customer` objects. diff --git a/content/guides/testcontainers-java-getting-started/run-tests.md b/content/guides/testcontainers-java-getting-started/run-tests.md deleted file mode 100644 index 6ab956485a2d..000000000000 --- a/content/guides/testcontainers-java-getting-started/run-tests.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -title: Run tests and next steps -linkTitle: Run tests -description: Run your Testcontainers-based integration tests and explore next steps. -keywords: testcontainers, java, postgresql, integration testing, run tests -weight: 30 ---- - -## Run the tests - -Run the tests using Maven: - -```console -$ mvn test -``` - -You can see in the logs that Testcontainers pulls the Postgres Docker image -from Docker Hub (if not already available locally), starts the container, and -runs the test. - -Writing an integration test using Testcontainers works like writing a unit test -that you can run from your IDE. Your teammates can clone the project -and run tests without installing Postgres on their machines. - -## Summary - -The Testcontainers for Java library helps you write integration tests using the -same type of database (Postgres) that you use in production, instead of mocks. -Because you aren't using mocks and instead talk to real services, you're free -to refactor code and still verify that the application works as expected. - -In addition to Postgres, Testcontainers provides dedicated modules for many -SQL databases, NoSQL databases, messaging queues, and more. You can use -Testcontainers to run any containerized dependency for your tests. - -To learn more about Testcontainers, visit the -[Testcontainers overview](https://testcontainers.com/getting-started/). - -## Further reading - -- [Testcontainers container lifecycle management using JUnit 5](https://testcontainers.com/guides/testcontainers-container-lifecycle/) -- [Replace H2 with a real database for testing](https://testcontainers.com/guides/replace-h2-with-real-database-for-testing/) -- [Getting started with Testcontainers in a Java Spring Boot project](https://testcontainers.com/guides/testing-spring-boot-rest-api-using-testcontainers/) diff --git a/content/guides/testcontainers-java-getting-started/write-tests.md b/content/guides/testcontainers-java-getting-started/write-tests.md deleted file mode 100644 index 051d5bc8a591..000000000000 --- a/content/guides/testcontainers-java-getting-started/write-tests.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -title: Write tests with Testcontainers -linkTitle: Write tests -description: Write your first integration test using Testcontainers for Java and PostgreSQL. -keywords: testcontainers, java, postgresql, junit, integration testing -weight: 20 ---- - -You have the `CustomerService` implementation ready, but for testing you need a -PostgreSQL database. You can use Testcontainers to spin up a Postgres database -in a Docker container and run your tests against it. - -## Add Testcontainers dependencies - -Add the Testcontainers PostgreSQL module as a test dependency in `pom.xml`: - -```xml - - org.testcontainers - testcontainers-postgresql - 2.0.4 - test - -``` - -Since the application uses a Postgres database, the Testcontainers Postgres -module provides a `PostgreSQLContainer` class for managing the container. - -## Write the test - -Create `CustomerServiceTest.java` under `src/test/java`: - -```java -package com.testcontainers.demo; - -import static org.junit.jupiter.api.Assertions.assertEquals; - -import java.util.List; -import org.junit.jupiter.api.AfterAll; -import org.junit.jupiter.api.BeforeAll; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.Test; -import org.testcontainers.postgresql.PostgreSQLContainer; - -class CustomerServiceTest { - - static PostgreSQLContainer postgres = new PostgreSQLContainer( - "postgres:16-alpine" - ); - - CustomerService customerService; - - @BeforeAll - static void beforeAll() { - postgres.start(); - } - - @AfterAll - static void afterAll() { - postgres.stop(); - } - - @BeforeEach - void setUp() { - DBConnectionProvider connectionProvider = new DBConnectionProvider( - postgres.getJdbcUrl(), - postgres.getUsername(), - postgres.getPassword() - ); - customerService = new CustomerService(connectionProvider); - } - - @Test - void shouldGetCustomers() { - customerService.createCustomer(new Customer(1L, "George")); - customerService.createCustomer(new Customer(2L, "John")); - - List customers = customerService.getAllCustomers(); - assertEquals(2, customers.size()); - } -} -``` - -Here's what the test does: - -- Declares a `PostgreSQLContainer` with the `postgres:16-alpine` Docker image. -- The `@BeforeAll` callback starts the Postgres container before any test - methods run. -- The `@BeforeEach` callback creates a `DBConnectionProvider` using the JDBC - connection parameters from the container, then creates a `CustomerService`. - The `CustomerService` constructor creates the `customers` table if it - doesn't exist. -- `shouldGetCustomers()` inserts 2 customer records, fetches all customers, - and asserts the count. -- The `@AfterAll` callback stops the container after all test methods finish. diff --git a/content/guides/testcontainers-java-jooq-flyway/_index.md b/content/guides/testcontainers-java-jooq-flyway/_index.md index 86e12078d0df..e6a3a954eb34 100644 --- a/content/guides/testcontainers-java-jooq-flyway/_index.md +++ b/content/guides/testcontainers-java-jooq-flyway/_index.md @@ -6,14 +6,16 @@ keywords: testcontainers, java, testing, jooq, flyway, postgresql, spring boot, summary: | Generate typesafe jOOQ code from a real PostgreSQL database managed by Flyway migrations, then test repositories using Testcontainers. -toc_min: 1 -toc_max: 2 -tags: [testing-with-docker] -languages: [java] +aliases: + - /guides/testcontainers-java-jooq-flyway/create-project/ + - /guides/testcontainers-java-jooq-flyway/run-tests/ + - /guides/testcontainers-java-jooq-flyway/write-tests/ params: + tags: [testing] time: 25 minutes --- + In this guide, you will learn how to: @@ -34,3 +36,575 @@ In this guide, you will learn how to: > If you're new to Testcontainers, visit the > [Testcontainers overview](https://testcontainers.com/getting-started/) to learn more about > Testcontainers and the benefits of using it. + +## Create the Spring Boot project + +### Set up the project + +Create a Spring Boot project from [Spring Initializr](https://start.spring.io) +by selecting Maven as the build tool and adding the **JOOQ Access Layer**, +**Flyway Migration**, **Spring Boot DevTools**, **PostgreSQL Driver**, and +**Testcontainers** starters. + +Alternatively, clone the +[guide repository](https://github.com/testcontainers/tc-guide-working-with-jooq-flyway-using-testcontainers). + +jOOQ (jOOQ Object Oriented Querying) provides a fluent API for building +typesafe SQL queries. To get the full benefit of its typesafe DSL, you need +to generate Java code from your database tables, views, and other objects. + +> [!TIP] +> To learn more about how the jOOQ code generator helps, read +> [Why You Should Use jOOQ With Code Generation](https://blog.jooq.org/why-you-should-use-jooq-with-code-generation/). + +The typical process for building and testing the application with jOOQ code +generation is: + +1. Create a database instance using Testcontainers. +2. Apply Flyway database migrations. +3. Run the jOOQ code generator to produce Java code from the database objects. +4. Run integration tests. + +The +[testcontainers-jooq-codegen-maven-plugin](https://github.com/testcontainers/testcontainers-jooq-codegen-maven-plugin) +automates this as part of the Maven build. + +### Create Flyway migration scripts + +The sample application has `users`, `posts`, and `comments` tables. Create +the first migration script following the Flyway naming convention. + +Create `src/main/resources/db/migration/V1__create_tables.sql`: + +```sql +create table users +( + id bigserial not null, + name varchar not null, + email varchar not null, + created_at timestamp, + updated_at timestamp, + primary key (id), + constraint user_email_unique unique (email) +); + +create table posts +( + id bigserial not null, + title varchar not null, + content varchar not null, + created_by bigint references users (id) not null, + created_at timestamp, + updated_at timestamp, + primary key (id) +); + +create table comments +( + id bigserial not null, + name varchar not null, + content varchar not null, + post_id bigint references posts (id) not null, + created_at timestamp, + updated_at timestamp, + primary key (id) +); + +ALTER SEQUENCE users_id_seq RESTART WITH 101; +ALTER SEQUENCE posts_id_seq RESTART WITH 101; +ALTER SEQUENCE comments_id_seq RESTART WITH 101; +``` + +The sequence values restart at 101 so that you can insert sample data with +explicit primary key values for testing. + +### Configure jOOQ code generation + +Add the `testcontainers-jooq-codegen-maven-plugin` to `pom.xml`: + +```xml + + 2.0.4 + 0.0.4 + + + + + + org.testcontainers + testcontainers-jooq-codegen-maven-plugin + ${testcontainers-jooq-codegen-maven-plugin.version} + + + org.testcontainers + testcontainers-postgresql + ${testcontainers.version} + + + org.postgresql + postgresql + ${postgresql.version} + + + + + generate-jooq-sources + + generate + + generate-sources + + + POSTGRES + postgres:16-alpine + + + + filesystem:src/main/resources/db/migration + + + + + + .* + flyway_schema_history + public + + + com.testcontainers.demo.jooq + target/generated-sources/jooq + + + + + + + + + +``` + +Here's what the plugin configuration does: + +- The `/` section sets the database type to + `POSTGRES` and the Docker image to `postgres:16-alpine`. +- The `/` section points to the Flyway migration + scripts. +- The `/` section configures the package name and + output directory for the generated code. You can use any configuration + option that the official `jooq-code-generator` plugin supports. + +When you run `./mvnw clean package`, the plugin uses Testcontainers to +spin up a PostgreSQL container, applies the Flyway migrations, and generates +Java code under `target/generated-sources/jooq`. + +### Create model classes + +Create model classes to represent the data structures for various use cases. +These records hold a subset of column values from the tables. + +`User.java`: + +```java +package com.testcontainers.demo.domain; + +public record User(Long id, String name, String email) {} +``` + +`Post.java`: + +```java +package com.testcontainers.demo.domain; + +import java.time.LocalDateTime; +import java.util.List; + +public record Post( + Long id, + String title, + String content, + User createdBy, + List comments, + LocalDateTime createdAt, + LocalDateTime updatedAt +) {} +``` + +`Comment.java`: + +```java +package com.testcontainers.demo.domain; + +import java.time.LocalDateTime; + +public record Comment( + Long id, + String name, + String content, + LocalDateTime createdAt, + LocalDateTime updatedAt +) {} +``` + +### Implement repositories using jOOQ + +Create `UserRepository.java` with methods to create a user and look up a user +by email: + +```java +package com.testcontainers.demo.domain; + +import static com.testcontainers.demo.jooq.tables.Users.USERS; +import static org.jooq.Records.mapping; + +import java.time.LocalDateTime; +import java.util.Optional; +import org.jooq.DSLContext; +import org.springframework.stereotype.Repository; + +@Repository +class UserRepository { + + private final DSLContext dsl; + + UserRepository(DSLContext dsl) { + this.dsl = dsl; + } + + public User createUser(User user) { + return this.dsl.insertInto(USERS) + .set(USERS.NAME, user.name()) + .set(USERS.EMAIL, user.email()) + .set(USERS.CREATED_AT, LocalDateTime.now()) + .returningResult(USERS.ID, USERS.NAME, USERS.EMAIL) + .fetchOne(mapping(User::new)); + } + + public Optional getUserByEmail(String email) { + return this.dsl.select(USERS.ID, USERS.NAME, USERS.EMAIL) + .from(USERS) + .where(USERS.EMAIL.equalIgnoreCase(email)) + .fetchOptional(mapping(User::new)); + } +} +``` + +The jOOQ DSL looks similar to SQL but written in Java. Because the code is +generated from the database schema, it stays in sync with the database +structure and provides type safety. For example, +`where(USERS.EMAIL.equalIgnoreCase(email))` expects a `String` value. If you +pass a non-string value like `123`, you get a compiler error. + +### Fetch complex object graphs + +jOOQ shines when it comes to complex queries. The database has a many-to-one +relationship from `Post` to `User` and a one-to-many relationship from `Post` +to `Comment`. + +Create `PostRepository.java` to load a `Post` with its creator and comments +using a single query with jOOQ's MULTISET feature: + +```java +package com.testcontainers.demo.domain; + +import static com.testcontainers.demo.jooq.Tables.COMMENTS; +import static com.testcontainers.demo.jooq.tables.Posts.POSTS; +import static org.jooq.Records.mapping; +import static org.jooq.impl.DSL.multiset; +import static org.jooq.impl.DSL.row; +import static org.jooq.impl.DSL.select; + +import java.util.Optional; +import org.jooq.DSLContext; +import org.springframework.stereotype.Repository; + +@Repository +class PostRepository { + + private final DSLContext dsl; + + PostRepository(DSLContext dsl) { + this.dsl = dsl; + } + + public Optional getPostById(Long id) { + return this.dsl.select( + POSTS.ID, + POSTS.TITLE, + POSTS.CONTENT, + row(POSTS.users().ID, POSTS.users().NAME, POSTS.users().EMAIL) + .mapping(User::new) + .as("createdBy"), + multiset( + select( + COMMENTS.ID, + COMMENTS.NAME, + COMMENTS.CONTENT, + COMMENTS.CREATED_AT, + COMMENTS.UPDATED_AT + ) + .from(COMMENTS) + .where(POSTS.ID.eq(COMMENTS.POST_ID)) + ) + .as("comments") + .convertFrom(r -> r.map(mapping(Comment::new))), + POSTS.CREATED_AT, + POSTS.UPDATED_AT + ) + .from(POSTS) + .where(POSTS.ID.eq(id)) + .fetchOptional(mapping(Post::new)); + } +} +``` + +This uses jOOQ's +[nested records](https://www.jooq.org/doc/latest/manual/sql-building/column-expressions/nested-records/) +for the many-to-one `Post`-to-`User` association and +[MULTISET](https://www.jooq.org/doc/latest/manual/sql-building/column-expressions/multiset-value-constructor/) +for the one-to-many `Post`-to-`Comment` association. + +## Write tests with Testcontainers + +Before writing the tests, create an SQL script to seed test data at +`src/test/resources/test-data.sql`: + +```sql +DELETE FROM comments; +DELETE FROM posts; +DELETE FROM users; + +INSERT INTO users(id, name, email) VALUES +(1, 'Siva', 'siva@gmail.com'), +(2, 'Oleg', 'oleg@gmail.com'); + +INSERT INTO posts(id, title, content, created_by, created_at) VALUES +(1, 'Post 1 Title', 'Post 1 content', 1, CURRENT_TIMESTAMP), +(2, 'Post 2 Title', 'Post 2 content', 2, CURRENT_TIMESTAMP); + +INSERT INTO comments(id, name, content, post_id, created_at) VALUES +(1, 'Ron', 'Comment 1', 1, CURRENT_TIMESTAMP), +(2, 'James', 'Comment 2', 1, CURRENT_TIMESTAMP), +(3, 'Robert', 'Comment 3', 2, CURRENT_TIMESTAMP); +``` + +### Test with the @JooqTest slice + +The `@JooqTest` annotation loads only the persistence layer components and +auto-configures jOOQ's `DSLContext`. Use the Testcontainers special JDBC URL +to start a Postgres container. + +Create `UserRepositoryJooqTest.java`: + +```java +package com.testcontainers.demo.domain; + +import static org.assertj.core.api.Assertions.assertThat; + +import org.jooq.DSLContext; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.autoconfigure.jooq.JooqTest; +import org.springframework.test.context.jdbc.Sql; + +@JooqTest( + properties = { + "spring.test.database.replace=none", + "spring.datasource.url=jdbc:tc:postgresql:16-alpine:///db", + } +) +@Sql("/test-data.sql") +class UserRepositoryJooqTest { + + @Autowired + DSLContext dsl; + + UserRepository repository; + + @BeforeEach + void setUp() { + this.repository = new UserRepository(dsl); + } + + @Test + void shouldCreateUserSuccessfully() { + User user = new User(null, "John", "john@gmail.com"); + + User savedUser = repository.createUser(user); + + assertThat(savedUser.id()).isNotNull(); + assertThat(savedUser.name()).isEqualTo("John"); + assertThat(savedUser.email()).isEqualTo("john@gmail.com"); + } + + @Test + void shouldGetUserByEmail() { + User user = repository.getUserByEmail("siva@gmail.com").orElseThrow(); + + assertThat(user.id()).isEqualTo(1L); + assertThat(user.name()).isEqualTo("Siva"); + assertThat(user.email()).isEqualTo("siva@gmail.com"); + } +} +``` + +Here's what the test does: + +- `@JooqTest` loads only the persistence layer and auto-configures + `DSLContext`. +- The Testcontainers special JDBC URL + (`jdbc:tc:postgresql:16-alpine:///db`) starts a PostgreSQL container + automatically. +- Because `flyway-core` is on the classpath, Spring Boot runs the Flyway + migrations from `src/main/resources/db/migration` on startup. +- `@Sql("/test-data.sql")` loads the test data before each test. +- The `UserRepository` is instantiated manually with the injected + `DSLContext`. + +### Integration test with @SpringBootTest + +For a full integration test, use `@SpringBootTest` with the Testcontainers +`@ServiceConnection` support introduced in Spring Boot 3.1. + +Create `UserRepositoryTest.java`: + +```java +package com.testcontainers.demo.domain; + +import static org.assertj.core.api.Assertions.assertThat; + +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.testcontainers.service.connection.ServiceConnection; +import org.springframework.test.context.jdbc.Sql; +import org.testcontainers.postgresql.PostgreSQLContainer; +import org.testcontainers.junit.jupiter.Container; +import org.testcontainers.junit.jupiter.Testcontainers; + +@SpringBootTest +@Sql("/test-data.sql") +@Testcontainers +class UserRepositoryTest { + + @Container + @ServiceConnection + static PostgreSQLContainer postgres = new PostgreSQLContainer( + "postgres:16-alpine" + ); + + @Autowired + UserRepository repository; + + @Test + void shouldCreateUserSuccessfully() { + User user = new User(null, "John", "john@gmail.com"); + + User savedUser = repository.createUser(user); + + assertThat(savedUser.id()).isNotNull(); + assertThat(savedUser.name()).isEqualTo("John"); + assertThat(savedUser.email()).isEqualTo("john@gmail.com"); + } + + @Test + void shouldGetUserByEmail() { + User user = repository.getUserByEmail("siva@gmail.com").orElseThrow(); + + assertThat(user.id()).isEqualTo(1L); + assertThat(user.name()).isEqualTo("Siva"); + assertThat(user.email()).isEqualTo("siva@gmail.com"); + } +} +``` + +Here's what the test does: + +- `@SpringBootTest` loads the entire application context, so + `UserRepository` is injected directly. +- `@Testcontainers` and `@Container` manage the PostgreSQL container + lifecycle. +- `@ServiceConnection` auto-configures the datasource properties from the + running container, replacing the need for `@DynamicPropertySource`. +- `@Sql("/test-data.sql")` initializes the test data. + +### Test PostRepository + +Test the `PostRepository` that fetches complex object graphs using the +Testcontainers special JDBC URL. + +Create `PostRepositoryTest.java`: + +```java +package com.testcontainers.demo.domain; + +import static org.assertj.core.api.Assertions.assertThat; + +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.test.context.jdbc.Sql; + +@SpringBootTest( + properties = { + "spring.test.database.replace=none", + "spring.datasource.url=jdbc:tc:postgresql:16-alpine:///db", + } +) +@Sql("/test-data.sql") +class PostRepositoryTest { + + @Autowired + PostRepository repository; + + @Test + void shouldGetPostById() { + Post post = repository.getPostById(1L).orElseThrow(); + + assertThat(post.id()).isEqualTo(1L); + assertThat(post.title()).isEqualTo("Post 1 Title"); + assertThat(post.content()).isEqualTo("Post 1 content"); + assertThat(post.createdBy().id()).isEqualTo(1L); + assertThat(post.createdBy().name()).isEqualTo("Siva"); + assertThat(post.createdBy().email()).isEqualTo("siva@gmail.com"); + assertThat(post.comments()).hasSize(2); + } +} +``` + +This test verifies that `getPostById` loads the post along with its creator +and comments in a single query using jOOQ's MULTISET feature. + +## Run tests and next steps + +### Run the tests + +```console +$ ./mvnw test +``` + +You should see the PostgreSQL Docker container start, jOOQ code generation +complete, and all tests pass. After the tests finish, the container stops and +is removed automatically. + +### Summary + +The Testcontainers library helps you generate Java code from the database +using the jOOQ code generator and test your persistence layer against the +same type of database (PostgreSQL) that you use in production, instead of +mocks or in-memory databases. + +Because the code is always generated from the database's current state, you +can be confident that your code stays in sync with database changes. You're +free to refactor and still verify that the application works as expected. + +To learn more about Testcontainers, visit the +[Testcontainers overview](https://testcontainers.com/getting-started/). + +### Further reading + +- [jOOQ documentation](https://www.jooq.org/) +- [jOOQ code generation](https://www.jooq.org/doc/latest/manual/code-generation/) +- [Spring Boot Testcontainers support](https://docs.spring.io/spring-boot/reference/testing/testcontainers.html) +- [Replace H2 with a real database for testing](/guides/testcontainers-java-replace-h2/) diff --git a/content/guides/testcontainers-java-jooq-flyway/create-project.md b/content/guides/testcontainers-java-jooq-flyway/create-project.md deleted file mode 100644 index 83adb5349938..000000000000 --- a/content/guides/testcontainers-java-jooq-flyway/create-project.md +++ /dev/null @@ -1,332 +0,0 @@ ---- -title: Create the Spring Boot project -linkTitle: Create the project -description: Set up a Spring Boot project with jOOQ, Flyway, PostgreSQL, and Testcontainers code generation. -keywords: testcontainers, java, spring boot, jooq, flyway, postgresql, project setup -weight: 10 ---- - -## Set up the project - -Create a Spring Boot project from [Spring Initializr](https://start.spring.io) -by selecting Maven as the build tool and adding the **JOOQ Access Layer**, -**Flyway Migration**, **Spring Boot DevTools**, **PostgreSQL Driver**, and -**Testcontainers** starters. - -Alternatively, clone the -[guide repository](https://github.com/testcontainers/tc-guide-working-with-jooq-flyway-using-testcontainers). - -jOOQ (jOOQ Object Oriented Querying) provides a fluent API for building -typesafe SQL queries. To get the full benefit of its typesafe DSL, you need -to generate Java code from your database tables, views, and other objects. - -> [!TIP] -> To learn more about how the jOOQ code generator helps, read -> [Why You Should Use jOOQ With Code Generation](https://blog.jooq.org/why-you-should-use-jooq-with-code-generation/). - -The typical process for building and testing the application with jOOQ code -generation is: - -1. Create a database instance using Testcontainers. -2. Apply Flyway database migrations. -3. Run the jOOQ code generator to produce Java code from the database objects. -4. Run integration tests. - -The -[testcontainers-jooq-codegen-maven-plugin](https://github.com/testcontainers/testcontainers-jooq-codegen-maven-plugin) -automates this as part of the Maven build. - -## Create Flyway migration scripts - -The sample application has `users`, `posts`, and `comments` tables. Create -the first migration script following the Flyway naming convention. - -Create `src/main/resources/db/migration/V1__create_tables.sql`: - -```sql -create table users -( - id bigserial not null, - name varchar not null, - email varchar not null, - created_at timestamp, - updated_at timestamp, - primary key (id), - constraint user_email_unique unique (email) -); - -create table posts -( - id bigserial not null, - title varchar not null, - content varchar not null, - created_by bigint references users (id) not null, - created_at timestamp, - updated_at timestamp, - primary key (id) -); - -create table comments -( - id bigserial not null, - name varchar not null, - content varchar not null, - post_id bigint references posts (id) not null, - created_at timestamp, - updated_at timestamp, - primary key (id) -); - -ALTER SEQUENCE users_id_seq RESTART WITH 101; -ALTER SEQUENCE posts_id_seq RESTART WITH 101; -ALTER SEQUENCE comments_id_seq RESTART WITH 101; -``` - -The sequence values restart at 101 so that you can insert sample data with -explicit primary key values for testing. - -## Configure jOOQ code generation - -Add the `testcontainers-jooq-codegen-maven-plugin` to `pom.xml`: - -```xml - - 2.0.4 - 0.0.4 - - - - - - org.testcontainers - testcontainers-jooq-codegen-maven-plugin - ${testcontainers-jooq-codegen-maven-plugin.version} - - - org.testcontainers - testcontainers-postgresql - ${testcontainers.version} - - - org.postgresql - postgresql - ${postgresql.version} - - - - - generate-jooq-sources - - generate - - generate-sources - - - POSTGRES - postgres:16-alpine - - - - filesystem:src/main/resources/db/migration - - - - - - .* - flyway_schema_history - public - - - com.testcontainers.demo.jooq - target/generated-sources/jooq - - - - - - - - - -``` - -Here's what the plugin configuration does: - -- The `/` section sets the database type to - `POSTGRES` and the Docker image to `postgres:16-alpine`. -- The `/` section points to the Flyway migration - scripts. -- The `/` section configures the package name and - output directory for the generated code. You can use any configuration - option that the official `jooq-code-generator` plugin supports. - -When you run `./mvnw clean package`, the plugin uses Testcontainers to -spin up a PostgreSQL container, applies the Flyway migrations, and generates -Java code under `target/generated-sources/jooq`. - -## Create model classes - -Create model classes to represent the data structures for various use cases. -These records hold a subset of column values from the tables. - -`User.java`: - -```java -package com.testcontainers.demo.domain; - -public record User(Long id, String name, String email) {} -``` - -`Post.java`: - -```java -package com.testcontainers.demo.domain; - -import java.time.LocalDateTime; -import java.util.List; - -public record Post( - Long id, - String title, - String content, - User createdBy, - List comments, - LocalDateTime createdAt, - LocalDateTime updatedAt -) {} -``` - -`Comment.java`: - -```java -package com.testcontainers.demo.domain; - -import java.time.LocalDateTime; - -public record Comment( - Long id, - String name, - String content, - LocalDateTime createdAt, - LocalDateTime updatedAt -) {} -``` - -## Implement repositories using jOOQ - -Create `UserRepository.java` with methods to create a user and look up a user -by email: - -```java -package com.testcontainers.demo.domain; - -import static com.testcontainers.demo.jooq.tables.Users.USERS; -import static org.jooq.Records.mapping; - -import java.time.LocalDateTime; -import java.util.Optional; -import org.jooq.DSLContext; -import org.springframework.stereotype.Repository; - -@Repository -class UserRepository { - - private final DSLContext dsl; - - UserRepository(DSLContext dsl) { - this.dsl = dsl; - } - - public User createUser(User user) { - return this.dsl.insertInto(USERS) - .set(USERS.NAME, user.name()) - .set(USERS.EMAIL, user.email()) - .set(USERS.CREATED_AT, LocalDateTime.now()) - .returningResult(USERS.ID, USERS.NAME, USERS.EMAIL) - .fetchOne(mapping(User::new)); - } - - public Optional getUserByEmail(String email) { - return this.dsl.select(USERS.ID, USERS.NAME, USERS.EMAIL) - .from(USERS) - .where(USERS.EMAIL.equalIgnoreCase(email)) - .fetchOptional(mapping(User::new)); - } -} -``` - -The jOOQ DSL looks similar to SQL but written in Java. Because the code is -generated from the database schema, it stays in sync with the database -structure and provides type safety. For example, -`where(USERS.EMAIL.equalIgnoreCase(email))` expects a `String` value. If you -pass a non-string value like `123`, you get a compiler error. - -## Fetch complex object graphs - -jOOQ shines when it comes to complex queries. The database has a many-to-one -relationship from `Post` to `User` and a one-to-many relationship from `Post` -to `Comment`. - -Create `PostRepository.java` to load a `Post` with its creator and comments -using a single query with jOOQ's MULTISET feature: - -```java -package com.testcontainers.demo.domain; - -import static com.testcontainers.demo.jooq.Tables.COMMENTS; -import static com.testcontainers.demo.jooq.tables.Posts.POSTS; -import static org.jooq.Records.mapping; -import static org.jooq.impl.DSL.multiset; -import static org.jooq.impl.DSL.row; -import static org.jooq.impl.DSL.select; - -import java.util.Optional; -import org.jooq.DSLContext; -import org.springframework.stereotype.Repository; - -@Repository -class PostRepository { - - private final DSLContext dsl; - - PostRepository(DSLContext dsl) { - this.dsl = dsl; - } - - public Optional getPostById(Long id) { - return this.dsl.select( - POSTS.ID, - POSTS.TITLE, - POSTS.CONTENT, - row(POSTS.users().ID, POSTS.users().NAME, POSTS.users().EMAIL) - .mapping(User::new) - .as("createdBy"), - multiset( - select( - COMMENTS.ID, - COMMENTS.NAME, - COMMENTS.CONTENT, - COMMENTS.CREATED_AT, - COMMENTS.UPDATED_AT - ) - .from(COMMENTS) - .where(POSTS.ID.eq(COMMENTS.POST_ID)) - ) - .as("comments") - .convertFrom(r -> r.map(mapping(Comment::new))), - POSTS.CREATED_AT, - POSTS.UPDATED_AT - ) - .from(POSTS) - .where(POSTS.ID.eq(id)) - .fetchOptional(mapping(Post::new)); - } -} -``` - -This uses jOOQ's -[nested records](https://www.jooq.org/doc/latest/manual/sql-building/column-expressions/nested-records/) -for the many-to-one `Post`-to-`User` association and -[MULTISET](https://www.jooq.org/doc/latest/manual/sql-building/column-expressions/multiset-value-constructor/) -for the one-to-many `Post`-to-`Comment` association. diff --git a/content/guides/testcontainers-java-jooq-flyway/run-tests.md b/content/guides/testcontainers-java-jooq-flyway/run-tests.md deleted file mode 100644 index af346bffba04..000000000000 --- a/content/guides/testcontainers-java-jooq-flyway/run-tests.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -title: Run tests and next steps -linkTitle: Run tests -description: Run the jOOQ and Flyway integration tests and explore next steps. -keywords: testcontainers, java, spring boot, jooq, flyway, integration testing -weight: 30 ---- - -## Run the tests - -```console -$ ./mvnw test -``` - -You should see the PostgreSQL Docker container start, jOOQ code generation -complete, and all tests pass. After the tests finish, the container stops and -is removed automatically. - -## Summary - -The Testcontainers library helps you generate Java code from the database -using the jOOQ code generator and test your persistence layer against the -same type of database (PostgreSQL) that you use in production, instead of -mocks or in-memory databases. - -Because the code is always generated from the database's current state, you -can be confident that your code stays in sync with database changes. You're -free to refactor and still verify that the application works as expected. - -To learn more about Testcontainers, visit the -[Testcontainers overview](https://testcontainers.com/getting-started/). - -## Further reading - -- [jOOQ documentation](https://www.jooq.org/) -- [jOOQ code generation](https://www.jooq.org/doc/latest/manual/code-generation/) -- [Spring Boot Testcontainers support](https://docs.spring.io/spring-boot/reference/testing/testcontainers.html) -- [Replace H2 with a real database for testing](/guides/testcontainers-java-replace-h2/) diff --git a/content/guides/testcontainers-java-jooq-flyway/write-tests.md b/content/guides/testcontainers-java-jooq-flyway/write-tests.md deleted file mode 100644 index f4720256d295..000000000000 --- a/content/guides/testcontainers-java-jooq-flyway/write-tests.md +++ /dev/null @@ -1,217 +0,0 @@ ---- -title: Write tests with Testcontainers -linkTitle: Write tests -description: Test jOOQ repositories using Testcontainers with the @JooqTest slice and @SpringBootTest. -keywords: testcontainers, java, spring boot, jooq, flyway, postgresql, integration testing -weight: 20 ---- - -Before writing the tests, create an SQL script to seed test data at -`src/test/resources/test-data.sql`: - -```sql -DELETE FROM comments; -DELETE FROM posts; -DELETE FROM users; - -INSERT INTO users(id, name, email) VALUES -(1, 'Siva', 'siva@gmail.com'), -(2, 'Oleg', 'oleg@gmail.com'); - -INSERT INTO posts(id, title, content, created_by, created_at) VALUES -(1, 'Post 1 Title', 'Post 1 content', 1, CURRENT_TIMESTAMP), -(2, 'Post 2 Title', 'Post 2 content', 2, CURRENT_TIMESTAMP); - -INSERT INTO comments(id, name, content, post_id, created_at) VALUES -(1, 'Ron', 'Comment 1', 1, CURRENT_TIMESTAMP), -(2, 'James', 'Comment 2', 1, CURRENT_TIMESTAMP), -(3, 'Robert', 'Comment 3', 2, CURRENT_TIMESTAMP); -``` - -## Test with the @JooqTest slice - -The `@JooqTest` annotation loads only the persistence layer components and -auto-configures jOOQ's `DSLContext`. Use the Testcontainers special JDBC URL -to start a Postgres container. - -Create `UserRepositoryJooqTest.java`: - -```java -package com.testcontainers.demo.domain; - -import static org.assertj.core.api.Assertions.assertThat; - -import org.jooq.DSLContext; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.Test; -import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.test.autoconfigure.jooq.JooqTest; -import org.springframework.test.context.jdbc.Sql; - -@JooqTest( - properties = { - "spring.test.database.replace=none", - "spring.datasource.url=jdbc:tc:postgresql:16-alpine:///db", - } -) -@Sql("/test-data.sql") -class UserRepositoryJooqTest { - - @Autowired - DSLContext dsl; - - UserRepository repository; - - @BeforeEach - void setUp() { - this.repository = new UserRepository(dsl); - } - - @Test - void shouldCreateUserSuccessfully() { - User user = new User(null, "John", "john@gmail.com"); - - User savedUser = repository.createUser(user); - - assertThat(savedUser.id()).isNotNull(); - assertThat(savedUser.name()).isEqualTo("John"); - assertThat(savedUser.email()).isEqualTo("john@gmail.com"); - } - - @Test - void shouldGetUserByEmail() { - User user = repository.getUserByEmail("siva@gmail.com").orElseThrow(); - - assertThat(user.id()).isEqualTo(1L); - assertThat(user.name()).isEqualTo("Siva"); - assertThat(user.email()).isEqualTo("siva@gmail.com"); - } -} -``` - -Here's what the test does: - -- `@JooqTest` loads only the persistence layer and auto-configures - `DSLContext`. -- The Testcontainers special JDBC URL - (`jdbc:tc:postgresql:16-alpine:///db`) starts a PostgreSQL container - automatically. -- Because `flyway-core` is on the classpath, Spring Boot runs the Flyway - migrations from `src/main/resources/db/migration` on startup. -- `@Sql("/test-data.sql")` loads the test data before each test. -- The `UserRepository` is instantiated manually with the injected - `DSLContext`. - -## Integration test with @SpringBootTest - -For a full integration test, use `@SpringBootTest` with the Testcontainers -`@ServiceConnection` support introduced in Spring Boot 3.1. - -Create `UserRepositoryTest.java`: - -```java -package com.testcontainers.demo.domain; - -import static org.assertj.core.api.Assertions.assertThat; - -import org.junit.jupiter.api.Test; -import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.testcontainers.service.connection.ServiceConnection; -import org.springframework.test.context.jdbc.Sql; -import org.testcontainers.postgresql.PostgreSQLContainer; -import org.testcontainers.junit.jupiter.Container; -import org.testcontainers.junit.jupiter.Testcontainers; - -@SpringBootTest -@Sql("/test-data.sql") -@Testcontainers -class UserRepositoryTest { - - @Container - @ServiceConnection - static PostgreSQLContainer postgres = new PostgreSQLContainer( - "postgres:16-alpine" - ); - - @Autowired - UserRepository repository; - - @Test - void shouldCreateUserSuccessfully() { - User user = new User(null, "John", "john@gmail.com"); - - User savedUser = repository.createUser(user); - - assertThat(savedUser.id()).isNotNull(); - assertThat(savedUser.name()).isEqualTo("John"); - assertThat(savedUser.email()).isEqualTo("john@gmail.com"); - } - - @Test - void shouldGetUserByEmail() { - User user = repository.getUserByEmail("siva@gmail.com").orElseThrow(); - - assertThat(user.id()).isEqualTo(1L); - assertThat(user.name()).isEqualTo("Siva"); - assertThat(user.email()).isEqualTo("siva@gmail.com"); - } -} -``` - -Here's what the test does: - -- `@SpringBootTest` loads the entire application context, so - `UserRepository` is injected directly. -- `@Testcontainers` and `@Container` manage the PostgreSQL container - lifecycle. -- `@ServiceConnection` auto-configures the datasource properties from the - running container, replacing the need for `@DynamicPropertySource`. -- `@Sql("/test-data.sql")` initializes the test data. - -## Test PostRepository - -Test the `PostRepository` that fetches complex object graphs using the -Testcontainers special JDBC URL. - -Create `PostRepositoryTest.java`: - -```java -package com.testcontainers.demo.domain; - -import static org.assertj.core.api.Assertions.assertThat; - -import org.junit.jupiter.api.Test; -import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.test.context.jdbc.Sql; - -@SpringBootTest( - properties = { - "spring.test.database.replace=none", - "spring.datasource.url=jdbc:tc:postgresql:16-alpine:///db", - } -) -@Sql("/test-data.sql") -class PostRepositoryTest { - - @Autowired - PostRepository repository; - - @Test - void shouldGetPostById() { - Post post = repository.getPostById(1L).orElseThrow(); - - assertThat(post.id()).isEqualTo(1L); - assertThat(post.title()).isEqualTo("Post 1 Title"); - assertThat(post.content()).isEqualTo("Post 1 content"); - assertThat(post.createdBy().id()).isEqualTo(1L); - assertThat(post.createdBy().name()).isEqualTo("Siva"); - assertThat(post.createdBy().email()).isEqualTo("siva@gmail.com"); - assertThat(post.comments()).hasSize(2); - } -} -``` - -This test verifies that `getPostById` loads the post along with its creator -and comments in a single query using jOOQ's MULTISET feature. diff --git a/content/guides/testcontainers-java-keycloak-spring-boot/_index.md b/content/guides/testcontainers-java-keycloak-spring-boot/_index.md index 619b69146fa4..187f024cd3b6 100644 --- a/content/guides/testcontainers-java-keycloak-spring-boot/_index.md +++ b/content/guides/testcontainers-java-keycloak-spring-boot/_index.md @@ -6,14 +6,16 @@ keywords: testcontainers, java, spring boot, testing, keycloak, security, oauth2 summary: | Learn how to create an OAuth 2.0 Resource Server using Spring Boot, secure API endpoints with Keycloak, and test the application using the Testcontainers Keycloak module. -toc_min: 1 -toc_max: 2 -tags: [testing-with-docker] -languages: [java] +aliases: + - /guides/testcontainers-java-keycloak-spring-boot/create-project/ + - /guides/testcontainers-java-keycloak-spring-boot/run-tests/ + - /guides/testcontainers-java-keycloak-spring-boot/write-tests/ params: + tags: [testing] time: 30 minutes --- + In this guide, you'll learn how to: @@ -33,3 +35,571 @@ In this guide, you'll learn how to: > If you're new to Testcontainers, visit the > [Testcontainers overview](https://testcontainers.com/getting-started/) to learn more about > Testcontainers and the benefits of using it. + +## Create the Spring Boot project + +### Set up the project + +Create a Spring Boot project from [Spring Initializr](https://start.spring.io) +by selecting the **Spring Web**, **Validation**, **JDBC API**, +**PostgreSQL Driver**, **Spring Security**, **OAuth2 Resource Server**, and +**Testcontainers** starters. + +Alternatively, clone the +[guide repository](https://github.com/testcontainers/tc-guide-securing-spring-boot-microservice-using-keycloak-and-testcontainers). + +After generating the application, add the +[testcontainers-keycloak](https://github.com/dasniko/testcontainers-keycloak) +community module and [REST Assured](https://rest-assured.io/) as test +dependencies. + +The key dependencies in `pom.xml` are: + +```xml + + 17 + 2.0.4 + + + + org.springframework.boot + spring-boot-starter-web + + + org.springframework.boot + spring-boot-starter-validation + + + org.springframework.boot + spring-boot-starter-jdbc + + + org.postgresql + postgresql + runtime + + + org.springframework.boot + spring-boot-starter-security + + + org.springframework.boot + spring-boot-starter-oauth2-resource-server + + + org.springframework.boot + spring-boot-starter-test + test + + + org.springframework.security + spring-security-test + test + + + org.springframework.boot + spring-boot-testcontainers + test + + + org.testcontainers + testcontainers-junit-jupiter + test + + + org.testcontainers + testcontainers-postgresql + test + + + com.github.dasniko + testcontainers-keycloak + 3.4.0 + test + + + io.rest-assured + rest-assured + test + + +``` + +### Create the domain model + +Create a `Product` record that represents the domain object: + +```java +package com.testcontainers.products.domain; + +import jakarta.validation.constraints.NotEmpty; + +public record Product(Long id, @NotEmpty String title, String description) {} +``` + +### Create the repository + +Implement `ProductRepository` using Spring `JdbcClient` to interact with a +PostgreSQL database: + +```java +package com.testcontainers.products.domain; + +import java.util.List; +import org.springframework.jdbc.core.simple.JdbcClient; +import org.springframework.jdbc.support.GeneratedKeyHolder; +import org.springframework.jdbc.support.KeyHolder; +import org.springframework.stereotype.Repository; + +@Repository +public class ProductRepository { + + private final JdbcClient jdbcClient; + + public ProductRepository(JdbcClient jdbcClient) { + this.jdbcClient = jdbcClient; + } + + public List getAll() { + return jdbcClient.sql("SELECT * FROM products").query(Product.class).list(); + } + + public Product create(Product product) { + String sql = + "INSERT INTO products(title, description) VALUES (:title,:description) RETURNING id"; + KeyHolder keyHolder = new GeneratedKeyHolder(); + jdbcClient + .sql(sql) + .param("title", product.title()) + .param("description", product.description()) + .update(keyHolder); + Long id = keyHolder.getKeyAs(Long.class); + return new Product(id, product.title(), product.description()); + } +} +``` + +### Add a schema creation script + +Create `src/main/resources/schema.sql` to initialize the `products` table: + +```sql +CREATE TABLE products ( + id bigserial primary key, + title varchar not null, + description text +); +``` + +Enable schema initialization in `src/main/resources/application.properties`: + +```properties +spring.sql.init.mode=always +``` + +For production applications, use a database migration tool like Flyway or +Liquibase instead. + +### Implement the API endpoints + +Create `ProductController` with endpoints to fetch all products and create a +product: + +```java +package com.testcontainers.products.api; + +import com.testcontainers.products.domain.Product; +import com.testcontainers.products.domain.ProductRepository; +import jakarta.validation.Valid; +import java.util.List; +import org.springframework.http.HttpStatus; +import org.springframework.web.bind.annotation.GetMapping; +import org.springframework.web.bind.annotation.PostMapping; +import org.springframework.web.bind.annotation.RequestBody; +import org.springframework.web.bind.annotation.RequestMapping; +import org.springframework.web.bind.annotation.ResponseStatus; +import org.springframework.web.bind.annotation.RestController; + +@RestController +@RequestMapping("/api/products") +class ProductController { + + private final ProductRepository productRepository; + + ProductController(ProductRepository productRepository) { + this.productRepository = productRepository; + } + + @GetMapping + List getAll() { + return productRepository.getAll(); + } + + @PostMapping + @ResponseStatus(HttpStatus.CREATED) + Product createProduct(@RequestBody @Valid Product product) { + return productRepository.create(product); + } +} +``` + +### Configure OAuth 2.0 security + +Create a `SecurityConfig` class that protects the API endpoints using JWT +token-based authentication: + +```java +package com.testcontainers.products.config; + +import static org.springframework.security.config.Customizer.withDefaults; + +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.http.HttpMethod; +import org.springframework.security.config.annotation.web.builders.HttpSecurity; +import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity; +import org.springframework.security.config.annotation.web.configurers.CorsConfigurer; +import org.springframework.security.config.annotation.web.configurers.CsrfConfigurer; +import org.springframework.security.config.http.SessionCreationPolicy; +import org.springframework.security.web.SecurityFilterChain; + +@Configuration +@EnableWebSecurity +class SecurityConfig { + + @Bean + SecurityFilterChain securityFilterChain(HttpSecurity http) throws Exception { + http + .authorizeHttpRequests(c -> + c + .requestMatchers(HttpMethod.GET, "/api/products") + .permitAll() + .requestMatchers(HttpMethod.POST, "/api/products") + .authenticated() + .anyRequest() + .authenticated() + ) + .sessionManagement(c -> + c.sessionCreationPolicy(SessionCreationPolicy.STATELESS) + ) + .cors(CorsConfigurer::disable) + .csrf(CsrfConfigurer::disable) + .oauth2ResourceServer(oauth2 -> oauth2.jwt(withDefaults())); + return http.build(); + } +} +``` + +This configuration: + +- Permits unauthenticated access to `GET /api/products`. +- Requires authentication for `POST /api/products` and all other endpoints. +- Configures the OAuth 2.0 Resource Server with JWT token-based authentication. +- Disables CORS and CSRF because this is a stateless API. + +Add the JWT issuer URI to `application.properties`: + +```properties +spring.security.oauth2.resourceserver.jwt.issuer-uri=http://localhost:9090/realms/keycloaktcdemo +``` + +### Export the Keycloak realm configuration + +Before writing the tests, export a Keycloak realm configuration so that the test +environment can import it automatically. Start a temporary Keycloak instance: + +```console +$ docker run -p 9090:8080 \ + -e KEYCLOAK_ADMIN=admin \ + -e KEYCLOAK_ADMIN_PASSWORD=admin \ + quay.io/keycloak/keycloak:25 start-dev +``` + +Open `http://localhost:9090` and sign in to the Admin Console with `admin/admin`. +Then set up the realm: + +1. In the top-left corner, select the realm drop-down and create a realm named + `keycloaktcdemo`. +2. Under the `keycloaktcdemo` realm, create a client with the following + settings: + - **Client ID**: `product-service` + - **Client Authentication**: `On` + - **Authentication flow**: select only **Service accounts roles** +3. On the **Client details** screen, go to the **Credentials** tab and copy the + **Client secret** value. + +Export the realm configuration: + +```console +$ docker ps +# copy the keycloak container id + +$ docker exec -it /bin/bash + +$ /opt/keycloak/bin/kc.sh export --dir /opt/keycloak/data/import --realm keycloaktcdemo + +$ exit + +$ docker cp :/opt/keycloak/data/import/keycloaktcdemo-realm.json keycloaktcdemo-realm.json +``` + +Copy the exported `keycloaktcdemo-realm.json` file into `src/test/resources`. + +## Write tests with Testcontainers + +To test the secured API endpoints, you need a running Keycloak instance and a +PostgreSQL database, plus a started Spring context. Testcontainers spins up both +services in Docker containers and connects them to Spring through dynamic +property registration. + +### Configure the test containers + +Spring Boot's Testcontainers support lets you declare containers as beans. For +Keycloak, `@ServiceConnection` isn't available, but you can use +`DynamicPropertyRegistry` to set the JWT issuer URI dynamically. + +Create `ContainersConfig.java` under `src/test/java`: + +```java +package com.testcontainers.products; + +import dasniko.testcontainers.keycloak.KeycloakContainer; +import org.springframework.boot.test.context.TestConfiguration; +import org.springframework.boot.testcontainers.service.connection.ServiceConnection; +import org.springframework.context.annotation.Bean; +import org.springframework.test.context.DynamicPropertyRegistry; +import org.testcontainers.postgresql.PostgreSQLContainer; + +@TestConfiguration(proxyBeanMethods = false) +public class ContainersConfig { + + static String POSTGRES_IMAGE = "postgres:16-alpine"; + static String KEYCLOAK_IMAGE = "quay.io/keycloak/keycloak:25.0"; + static String realmImportFile = "/keycloaktcdemo-realm.json"; + static String realmName = "keycloaktcdemo"; + + @Bean + @ServiceConnection + PostgreSQLContainer postgres() { + return new PostgreSQLContainer(POSTGRES_IMAGE); + } + + @Bean + KeycloakContainer keycloak(DynamicPropertyRegistry registry) { + var keycloak = new KeycloakContainer(KEYCLOAK_IMAGE) + .withRealmImportFile(realmImportFile); + registry.add( + "spring.security.oauth2.resourceserver.jwt.issuer-uri", + () -> keycloak.getAuthServerUrl() + "/realms/" + realmName + ); + return keycloak; + } +} +``` + +This configuration: + +- Declares a `PostgreSQLContainer` bean with `@ServiceConnection`, which starts + a PostgreSQL container and automatically registers the datasource properties. +- Declares a `KeycloakContainer` bean using the `quay.io/keycloak/keycloak:25.0` + image, imports the realm configuration file, and dynamically registers the JWT + issuer URI from the Keycloak container's auth server URL. + +### Write the test + +Create `ProductControllerTests.java`: + +```java +package com.testcontainers.products.api; + +import static io.restassured.RestAssured.given; +import static io.restassured.RestAssured.when; +import static java.util.Collections.singletonList; +import static org.springframework.boot.test.context.SpringBootTest.WebEnvironment.RANDOM_PORT; + +import com.fasterxml.jackson.annotation.JsonProperty; +import com.testcontainers.products.ContainersConfig; +import io.restassured.RestAssured; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.autoconfigure.security.oauth2.resource.OAuth2ResourceServerProperties; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.web.server.LocalServerPort; +import org.springframework.context.annotation.Import; +import org.springframework.http.HttpEntity; +import org.springframework.http.HttpHeaders; +import org.springframework.http.MediaType; +import org.springframework.util.LinkedMultiValueMap; +import org.springframework.util.MultiValueMap; +import org.springframework.web.client.RestTemplate; + +@SpringBootTest(webEnvironment = RANDOM_PORT) +@Import(ContainersConfig.class) +class ProductControllerTests { + + static final String GRANT_TYPE_CLIENT_CREDENTIALS = "client_credentials"; + static final String CLIENT_ID = "product-service"; + static final String CLIENT_SECRET = "jTJJqdzeCSt3DmypfHZa42vX8U9rQKZ9"; + + @LocalServerPort + private int port; + + @Autowired + OAuth2ResourceServerProperties oAuth2ResourceServerProperties; + + @BeforeEach + void setup() { + RestAssured.port = port; + } + + @Test + void shouldGetProductsWithoutAuthToken() { + when().get("/api/products").then().statusCode(200); + } + + @Test + void shouldGetUnauthorizedWhenCreateProductWithoutAuthToken() { + given() + .contentType("application/json") + .body( + """ + { + "title": "New Product", + "description": "Brand New Product" + } + """ + ) + .when() + .post("/api/products") + .then() + .statusCode(401); + } + + @Test + void shouldCreateProductWithAuthToken() { + String token = getToken(); + + given() + .header("Authorization", "Bearer " + token) + .contentType("application/json") + .body( + """ + { + "title": "New Product", + "description": "Brand New Product" + } + """ + ) + .when() + .post("/api/products") + .then() + .statusCode(201); + } + + private String getToken() { + RestTemplate restTemplate = new RestTemplate(); + HttpHeaders httpHeaders = new HttpHeaders(); + httpHeaders.setContentType(MediaType.APPLICATION_FORM_URLENCODED); + + MultiValueMap map = new LinkedMultiValueMap<>(); + map.put("grant_type", singletonList(GRANT_TYPE_CLIENT_CREDENTIALS)); + map.put("client_id", singletonList(CLIENT_ID)); + map.put("client_secret", singletonList(CLIENT_SECRET)); + + String authServerUrl = + oAuth2ResourceServerProperties.getJwt().getIssuerUri() + + "/protocol/openid-connect/token"; + + var request = new HttpEntity<>(map, httpHeaders); + KeyCloakToken token = restTemplate.postForObject( + authServerUrl, + request, + KeyCloakToken.class + ); + + assert token != null; + return token.accessToken(); + } + + record KeyCloakToken(@JsonProperty("access_token") String accessToken) {} +} +``` + +Here's what the tests cover: + +- `shouldGetProductsWithoutAuthToken()` invokes `GET /api/products` without an + `Authorization` header. Because this endpoint is configured to permit + unauthenticated access, the response status code is 200. +- `shouldGetUnauthorizedWhenCreateProductWithoutAuthToken()` invokes the secured + `POST /api/products` endpoint without an `Authorization` header and asserts + the response status code is 401 (Unauthorized). +- `shouldCreateProductWithAuthToken()` first obtains an `access_token` using the + Client Credentials flow. It then includes the token as a Bearer token in the + `Authorization` header when invoking `POST /api/products` and asserts the + response status code is 201 (Created). + +The `getToken()` helper method requests an access token from the Keycloak token +endpoint using the client ID and client secret that were configured in the +exported realm. + +### Use Testcontainers for local development + +Spring Boot's Testcontainers support also works for local development. Create +`TestApplication.java` under `src/test/java`: + +```java +package com.testcontainers.products; + +import org.springframework.boot.SpringApplication; + +public class TestApplication { + + public static void main(String[] args) { + SpringApplication + .from(Application::main) + .with(ContainersConfig.class) + .run(args); + } +} +``` + +Run `TestApplication.java` from your IDE instead of the main `Application.java`. +It starts the containers defined in `ContainersConfig` and configures the +application to use the dynamically registered properties, so you don't have to +install or configure PostgreSQL and Keycloak manually. + +## Run tests and next steps + +### Run the tests + +```console +$ ./mvnw test +``` + +Or with Gradle: + +```console +$ ./gradlew test +``` + +You should see the Keycloak and PostgreSQL Docker containers start with the +realm settings imported and the tests pass. After the tests finish, the +containers stop and are removed automatically. + +### Summary + +The Testcontainers Keycloak module lets you develop and test applications using a +real Keycloak server instead of mocks. Testing against a real OAuth 2.0 +provider that mirrors your production setup gives you more confidence in your +security configuration and token-based authentication flows. + +To learn more about Testcontainers, visit the +[Testcontainers overview](https://testcontainers.com/getting-started/). + +### Further reading + +- [Getting started with Testcontainers in a Java Spring Boot project](https://testcontainers.com/guides/testing-spring-boot-rest-api-using-testcontainers/) +- [Testcontainers Keycloak module](https://testcontainers.com/modules/keycloak/) +- [testcontainers-keycloak GitHub repository](https://github.com/dasniko/testcontainers-keycloak) +- [Spring Boot OAuth 2.0 Resource Server](https://docs.spring.io/spring-security/reference/servlet/oauth2/resource-server/index.html) diff --git a/content/guides/testcontainers-java-keycloak-spring-boot/create-project.md b/content/guides/testcontainers-java-keycloak-spring-boot/create-project.md deleted file mode 100644 index db7f07663290..000000000000 --- a/content/guides/testcontainers-java-keycloak-spring-boot/create-project.md +++ /dev/null @@ -1,314 +0,0 @@ ---- -title: Create the Spring Boot project -linkTitle: Create the project -description: Set up a Spring Boot OAuth 2.0 Resource Server with Keycloak, PostgreSQL, and Testcontainers. -keywords: testcontainers, java, spring boot, keycloak, oauth2, postgresql, project setup -weight: 10 ---- - -## Set up the project - -Create a Spring Boot project from [Spring Initializr](https://start.spring.io) -by selecting the **Spring Web**, **Validation**, **JDBC API**, -**PostgreSQL Driver**, **Spring Security**, **OAuth2 Resource Server**, and -**Testcontainers** starters. - -Alternatively, clone the -[guide repository](https://github.com/testcontainers/tc-guide-securing-spring-boot-microservice-using-keycloak-and-testcontainers). - -After generating the application, add the -[testcontainers-keycloak](https://github.com/dasniko/testcontainers-keycloak) -community module and [REST Assured](https://rest-assured.io/) as test -dependencies. - -The key dependencies in `pom.xml` are: - -```xml - - 17 - 2.0.4 - - - - org.springframework.boot - spring-boot-starter-web - - - org.springframework.boot - spring-boot-starter-validation - - - org.springframework.boot - spring-boot-starter-jdbc - - - org.postgresql - postgresql - runtime - - - org.springframework.boot - spring-boot-starter-security - - - org.springframework.boot - spring-boot-starter-oauth2-resource-server - - - org.springframework.boot - spring-boot-starter-test - test - - - org.springframework.security - spring-security-test - test - - - org.springframework.boot - spring-boot-testcontainers - test - - - org.testcontainers - testcontainers-junit-jupiter - test - - - org.testcontainers - testcontainers-postgresql - test - - - com.github.dasniko - testcontainers-keycloak - 3.4.0 - test - - - io.rest-assured - rest-assured - test - - -``` - -## Create the domain model - -Create a `Product` record that represents the domain object: - -```java -package com.testcontainers.products.domain; - -import jakarta.validation.constraints.NotEmpty; - -public record Product(Long id, @NotEmpty String title, String description) {} -``` - -## Create the repository - -Implement `ProductRepository` using Spring `JdbcClient` to interact with a -PostgreSQL database: - -```java -package com.testcontainers.products.domain; - -import java.util.List; -import org.springframework.jdbc.core.simple.JdbcClient; -import org.springframework.jdbc.support.GeneratedKeyHolder; -import org.springframework.jdbc.support.KeyHolder; -import org.springframework.stereotype.Repository; - -@Repository -public class ProductRepository { - - private final JdbcClient jdbcClient; - - public ProductRepository(JdbcClient jdbcClient) { - this.jdbcClient = jdbcClient; - } - - public List getAll() { - return jdbcClient.sql("SELECT * FROM products").query(Product.class).list(); - } - - public Product create(Product product) { - String sql = - "INSERT INTO products(title, description) VALUES (:title,:description) RETURNING id"; - KeyHolder keyHolder = new GeneratedKeyHolder(); - jdbcClient - .sql(sql) - .param("title", product.title()) - .param("description", product.description()) - .update(keyHolder); - Long id = keyHolder.getKeyAs(Long.class); - return new Product(id, product.title(), product.description()); - } -} -``` - -## Add a schema creation script - -Create `src/main/resources/schema.sql` to initialize the `products` table: - -```sql -CREATE TABLE products ( - id bigserial primary key, - title varchar not null, - description text -); -``` - -Enable schema initialization in `src/main/resources/application.properties`: - -```properties -spring.sql.init.mode=always -``` - -For production applications, use a database migration tool like Flyway or -Liquibase instead. - -## Implement the API endpoints - -Create `ProductController` with endpoints to fetch all products and create a -product: - -```java -package com.testcontainers.products.api; - -import com.testcontainers.products.domain.Product; -import com.testcontainers.products.domain.ProductRepository; -import jakarta.validation.Valid; -import java.util.List; -import org.springframework.http.HttpStatus; -import org.springframework.web.bind.annotation.GetMapping; -import org.springframework.web.bind.annotation.PostMapping; -import org.springframework.web.bind.annotation.RequestBody; -import org.springframework.web.bind.annotation.RequestMapping; -import org.springframework.web.bind.annotation.ResponseStatus; -import org.springframework.web.bind.annotation.RestController; - -@RestController -@RequestMapping("/api/products") -class ProductController { - - private final ProductRepository productRepository; - - ProductController(ProductRepository productRepository) { - this.productRepository = productRepository; - } - - @GetMapping - List getAll() { - return productRepository.getAll(); - } - - @PostMapping - @ResponseStatus(HttpStatus.CREATED) - Product createProduct(@RequestBody @Valid Product product) { - return productRepository.create(product); - } -} -``` - -## Configure OAuth 2.0 security - -Create a `SecurityConfig` class that protects the API endpoints using JWT -token-based authentication: - -```java -package com.testcontainers.products.config; - -import static org.springframework.security.config.Customizer.withDefaults; - -import org.springframework.context.annotation.Bean; -import org.springframework.context.annotation.Configuration; -import org.springframework.http.HttpMethod; -import org.springframework.security.config.annotation.web.builders.HttpSecurity; -import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity; -import org.springframework.security.config.annotation.web.configurers.CorsConfigurer; -import org.springframework.security.config.annotation.web.configurers.CsrfConfigurer; -import org.springframework.security.config.http.SessionCreationPolicy; -import org.springframework.security.web.SecurityFilterChain; - -@Configuration -@EnableWebSecurity -class SecurityConfig { - - @Bean - SecurityFilterChain securityFilterChain(HttpSecurity http) throws Exception { - http - .authorizeHttpRequests(c -> - c - .requestMatchers(HttpMethod.GET, "/api/products") - .permitAll() - .requestMatchers(HttpMethod.POST, "/api/products") - .authenticated() - .anyRequest() - .authenticated() - ) - .sessionManagement(c -> - c.sessionCreationPolicy(SessionCreationPolicy.STATELESS) - ) - .cors(CorsConfigurer::disable) - .csrf(CsrfConfigurer::disable) - .oauth2ResourceServer(oauth2 -> oauth2.jwt(withDefaults())); - return http.build(); - } -} -``` - -This configuration: - -- Permits unauthenticated access to `GET /api/products`. -- Requires authentication for `POST /api/products` and all other endpoints. -- Configures the OAuth 2.0 Resource Server with JWT token-based authentication. -- Disables CORS and CSRF because this is a stateless API. - -Add the JWT issuer URI to `application.properties`: - -```properties -spring.security.oauth2.resourceserver.jwt.issuer-uri=http://localhost:9090/realms/keycloaktcdemo -``` - -## Export the Keycloak realm configuration - -Before writing the tests, export a Keycloak realm configuration so that the test -environment can import it automatically. Start a temporary Keycloak instance: - -```console -$ docker run -p 9090:8080 \ - -e KEYCLOAK_ADMIN=admin \ - -e KEYCLOAK_ADMIN_PASSWORD=admin \ - quay.io/keycloak/keycloak:25 start-dev -``` - -Open `http://localhost:9090` and sign in to the Admin Console with `admin/admin`. -Then set up the realm: - -1. In the top-left corner, select the realm drop-down and create a realm named - `keycloaktcdemo`. -2. Under the `keycloaktcdemo` realm, create a client with the following - settings: - - **Client ID**: `product-service` - - **Client Authentication**: `On` - - **Authentication flow**: select only **Service accounts roles** -3. On the **Client details** screen, go to the **Credentials** tab and copy the - **Client secret** value. - -Export the realm configuration: - -```console -$ docker ps -# copy the keycloak container id - -$ docker exec -it /bin/bash - -$ /opt/keycloak/bin/kc.sh export --dir /opt/keycloak/data/import --realm keycloaktcdemo - -$ exit - -$ docker cp :/opt/keycloak/data/import/keycloaktcdemo-realm.json keycloaktcdemo-realm.json -``` - -Copy the exported `keycloaktcdemo-realm.json` file into `src/test/resources`. diff --git a/content/guides/testcontainers-java-keycloak-spring-boot/run-tests.md b/content/guides/testcontainers-java-keycloak-spring-boot/run-tests.md deleted file mode 100644 index 9f3a72f54ff7..000000000000 --- a/content/guides/testcontainers-java-keycloak-spring-boot/run-tests.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -title: Run tests and next steps -linkTitle: Run tests -description: Run your Testcontainers-based Spring Boot Keycloak integration tests and explore next steps. -keywords: testcontainers, java, spring boot, keycloak, oauth2, integration testing -weight: 30 ---- - -## Run the tests - -```console -$ ./mvnw test -``` - -Or with Gradle: - -```console -$ ./gradlew test -``` - -You should see the Keycloak and PostgreSQL Docker containers start with the -realm settings imported and the tests pass. After the tests finish, the -containers stop and are removed automatically. - -## Summary - -The Testcontainers Keycloak module lets you develop and test applications using a -real Keycloak server instead of mocks. Testing against a real OAuth 2.0 -provider that mirrors your production setup gives you more confidence in your -security configuration and token-based authentication flows. - -To learn more about Testcontainers, visit the -[Testcontainers overview](https://testcontainers.com/getting-started/). - -## Further reading - -- [Getting started with Testcontainers in a Java Spring Boot project](https://testcontainers.com/guides/testing-spring-boot-rest-api-using-testcontainers/) -- [Testcontainers Keycloak module](https://testcontainers.com/modules/keycloak/) -- [testcontainers-keycloak GitHub repository](https://github.com/dasniko/testcontainers-keycloak) -- [Spring Boot OAuth 2.0 Resource Server](https://docs.spring.io/spring-security/reference/servlet/oauth2/resource-server/index.html) diff --git a/content/guides/testcontainers-java-keycloak-spring-boot/write-tests.md b/content/guides/testcontainers-java-keycloak-spring-boot/write-tests.md deleted file mode 100644 index 90262b3a94a8..000000000000 --- a/content/guides/testcontainers-java-keycloak-spring-boot/write-tests.md +++ /dev/null @@ -1,229 +0,0 @@ ---- -title: Write tests with Testcontainers -linkTitle: Write tests -description: Test the secured Spring Boot API endpoints using Testcontainers Keycloak and PostgreSQL modules. -keywords: testcontainers, java, spring boot, keycloak, oauth2, postgresql, integration testing -weight: 20 ---- - -To test the secured API endpoints, you need a running Keycloak instance and a -PostgreSQL database, plus a started Spring context. Testcontainers spins up both -services in Docker containers and connects them to Spring through dynamic -property registration. - -## Configure the test containers - -Spring Boot's Testcontainers support lets you declare containers as beans. For -Keycloak, `@ServiceConnection` isn't available, but you can use -`DynamicPropertyRegistry` to set the JWT issuer URI dynamically. - -Create `ContainersConfig.java` under `src/test/java`: - -```java -package com.testcontainers.products; - -import dasniko.testcontainers.keycloak.KeycloakContainer; -import org.springframework.boot.test.context.TestConfiguration; -import org.springframework.boot.testcontainers.service.connection.ServiceConnection; -import org.springframework.context.annotation.Bean; -import org.springframework.test.context.DynamicPropertyRegistry; -import org.testcontainers.postgresql.PostgreSQLContainer; - -@TestConfiguration(proxyBeanMethods = false) -public class ContainersConfig { - - static String POSTGRES_IMAGE = "postgres:16-alpine"; - static String KEYCLOAK_IMAGE = "quay.io/keycloak/keycloak:25.0"; - static String realmImportFile = "/keycloaktcdemo-realm.json"; - static String realmName = "keycloaktcdemo"; - - @Bean - @ServiceConnection - PostgreSQLContainer postgres() { - return new PostgreSQLContainer(POSTGRES_IMAGE); - } - - @Bean - KeycloakContainer keycloak(DynamicPropertyRegistry registry) { - var keycloak = new KeycloakContainer(KEYCLOAK_IMAGE) - .withRealmImportFile(realmImportFile); - registry.add( - "spring.security.oauth2.resourceserver.jwt.issuer-uri", - () -> keycloak.getAuthServerUrl() + "/realms/" + realmName - ); - return keycloak; - } -} -``` - -This configuration: - -- Declares a `PostgreSQLContainer` bean with `@ServiceConnection`, which starts - a PostgreSQL container and automatically registers the datasource properties. -- Declares a `KeycloakContainer` bean using the `quay.io/keycloak/keycloak:25.0` - image, imports the realm configuration file, and dynamically registers the JWT - issuer URI from the Keycloak container's auth server URL. - -## Write the test - -Create `ProductControllerTests.java`: - -```java -package com.testcontainers.products.api; - -import static io.restassured.RestAssured.given; -import static io.restassured.RestAssured.when; -import static java.util.Collections.singletonList; -import static org.springframework.boot.test.context.SpringBootTest.WebEnvironment.RANDOM_PORT; - -import com.fasterxml.jackson.annotation.JsonProperty; -import com.testcontainers.products.ContainersConfig; -import io.restassured.RestAssured; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.Test; -import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.autoconfigure.security.oauth2.resource.OAuth2ResourceServerProperties; -import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.test.web.server.LocalServerPort; -import org.springframework.context.annotation.Import; -import org.springframework.http.HttpEntity; -import org.springframework.http.HttpHeaders; -import org.springframework.http.MediaType; -import org.springframework.util.LinkedMultiValueMap; -import org.springframework.util.MultiValueMap; -import org.springframework.web.client.RestTemplate; - -@SpringBootTest(webEnvironment = RANDOM_PORT) -@Import(ContainersConfig.class) -class ProductControllerTests { - - static final String GRANT_TYPE_CLIENT_CREDENTIALS = "client_credentials"; - static final String CLIENT_ID = "product-service"; - static final String CLIENT_SECRET = "jTJJqdzeCSt3DmypfHZa42vX8U9rQKZ9"; - - @LocalServerPort - private int port; - - @Autowired - OAuth2ResourceServerProperties oAuth2ResourceServerProperties; - - @BeforeEach - void setup() { - RestAssured.port = port; - } - - @Test - void shouldGetProductsWithoutAuthToken() { - when().get("/api/products").then().statusCode(200); - } - - @Test - void shouldGetUnauthorizedWhenCreateProductWithoutAuthToken() { - given() - .contentType("application/json") - .body( - """ - { - "title": "New Product", - "description": "Brand New Product" - } - """ - ) - .when() - .post("/api/products") - .then() - .statusCode(401); - } - - @Test - void shouldCreateProductWithAuthToken() { - String token = getToken(); - - given() - .header("Authorization", "Bearer " + token) - .contentType("application/json") - .body( - """ - { - "title": "New Product", - "description": "Brand New Product" - } - """ - ) - .when() - .post("/api/products") - .then() - .statusCode(201); - } - - private String getToken() { - RestTemplate restTemplate = new RestTemplate(); - HttpHeaders httpHeaders = new HttpHeaders(); - httpHeaders.setContentType(MediaType.APPLICATION_FORM_URLENCODED); - - MultiValueMap map = new LinkedMultiValueMap<>(); - map.put("grant_type", singletonList(GRANT_TYPE_CLIENT_CREDENTIALS)); - map.put("client_id", singletonList(CLIENT_ID)); - map.put("client_secret", singletonList(CLIENT_SECRET)); - - String authServerUrl = - oAuth2ResourceServerProperties.getJwt().getIssuerUri() + - "/protocol/openid-connect/token"; - - var request = new HttpEntity<>(map, httpHeaders); - KeyCloakToken token = restTemplate.postForObject( - authServerUrl, - request, - KeyCloakToken.class - ); - - assert token != null; - return token.accessToken(); - } - - record KeyCloakToken(@JsonProperty("access_token") String accessToken) {} -} -``` - -Here's what the tests cover: - -- `shouldGetProductsWithoutAuthToken()` invokes `GET /api/products` without an - `Authorization` header. Because this endpoint is configured to permit - unauthenticated access, the response status code is 200. -- `shouldGetUnauthorizedWhenCreateProductWithoutAuthToken()` invokes the secured - `POST /api/products` endpoint without an `Authorization` header and asserts - the response status code is 401 (Unauthorized). -- `shouldCreateProductWithAuthToken()` first obtains an `access_token` using the - Client Credentials flow. It then includes the token as a Bearer token in the - `Authorization` header when invoking `POST /api/products` and asserts the - response status code is 201 (Created). - -The `getToken()` helper method requests an access token from the Keycloak token -endpoint using the client ID and client secret that were configured in the -exported realm. - -## Use Testcontainers for local development - -Spring Boot's Testcontainers support also works for local development. Create -`TestApplication.java` under `src/test/java`: - -```java -package com.testcontainers.products; - -import org.springframework.boot.SpringApplication; - -public class TestApplication { - - public static void main(String[] args) { - SpringApplication - .from(Application::main) - .with(ContainersConfig.class) - .run(args); - } -} -``` - -Run `TestApplication.java` from your IDE instead of the main `Application.java`. -It starts the containers defined in `ContainersConfig` and configures the -application to use the dynamically registered properties, so you don't have to -install or configure PostgreSQL and Keycloak manually. diff --git a/content/guides/testcontainers-java-lifecycle/_index.md b/content/guides/testcontainers-java-lifecycle/_index.md index b24f7928ae93..2db26128fc9e 100644 --- a/content/guides/testcontainers-java-lifecycle/_index.md +++ b/content/guides/testcontainers-java-lifecycle/_index.md @@ -7,14 +7,17 @@ summary: | Learn different approaches to manage container lifecycle with Testcontainers using JUnit 5 lifecycle callbacks, extension annotations, and the singleton containers pattern. -toc_min: 1 -toc_max: 2 -tags: [testing-with-docker] -languages: [java] +aliases: + - /guides/testcontainers-java-lifecycle/create-project/ + - /guides/testcontainers-java-lifecycle/extension-annotations/ + - /guides/testcontainers-java-lifecycle/lifecycle-callbacks/ + - /guides/testcontainers-java-lifecycle/singleton-containers/ params: + tags: [testing] time: 20 minutes --- + In this guide, you will learn how to: @@ -34,3 +37,433 @@ In this guide, you will learn how to: > If you're new to Testcontainers, visit the > [Testcontainers overview](https://testcontainers.com/getting-started/) to learn more about > Testcontainers and the benefits of using it. + +## Create the project and business logic + +### Set up the project + +Create a Java project with Maven and add the required dependencies: + +```xml + + + org.postgresql + postgresql + 42.7.3 + + + ch.qos.logback + logback-classic + 1.5.6 + + + org.junit.jupiter + junit-jupiter + 5.10.2 + test + + + org.testcontainers + testcontainers-junit-jupiter + 2.0.4 + test + + + org.testcontainers + testcontainers-postgresql + 2.0.4 + test + + +``` + +### Create the business logic + +Create a `Customer` record: + +```java +package com.testcontainers.demo; + +public record Customer(Long id, String name) {} +``` + +Create a `CustomerService` class with methods to create, retrieve, and delete +customers: + +```java +package com.testcontainers.demo; + +import java.sql.Connection; +import java.sql.DriverManager; +import java.sql.PreparedStatement; +import java.sql.ResultSet; +import java.sql.SQLException; +import java.util.ArrayList; +import java.util.List; +import java.util.Optional; + +public class CustomerService { + + private final String url; + private final String username; + private final String password; + + public CustomerService(String url, String username, String password) { + this.url = url; + this.username = username; + this.password = password; + createCustomersTableIfNotExists(); + } + + public void createCustomer(Customer customer) { + try (Connection conn = this.getConnection()) { + PreparedStatement pstmt = conn.prepareStatement( + "insert into customers(id,name) values(?,?)" + ); + pstmt.setLong(1, customer.id()); + pstmt.setString(2, customer.name()); + pstmt.execute(); + } catch (SQLException e) { + throw new RuntimeException(e); + } + } + + public List getAllCustomers() { + List customers = new ArrayList<>(); + try (Connection conn = this.getConnection()) { + PreparedStatement pstmt = conn.prepareStatement( + "select id,name from customers" + ); + ResultSet rs = pstmt.executeQuery(); + while (rs.next()) { + long id = rs.getLong("id"); + String name = rs.getString("name"); + customers.add(new Customer(id, name)); + } + } catch (SQLException e) { + throw new RuntimeException(e); + } + return customers; + } + + public Optional getCustomer(Long customerId) { + try (Connection conn = this.getConnection()) { + PreparedStatement pstmt = conn.prepareStatement( + "select id,name from customers where id = ?" + ); + pstmt.setLong(1, customerId); + ResultSet rs = pstmt.executeQuery(); + if (rs.next()) { + long id = rs.getLong("id"); + String name = rs.getString("name"); + return Optional.of(new Customer(id, name)); + } + } catch (SQLException e) { + throw new RuntimeException(e); + } + return Optional.empty(); + } + + public void deleteAllCustomers() { + try (Connection conn = this.getConnection()) { + PreparedStatement pstmt = conn.prepareStatement("delete from customers"); + pstmt.execute(); + } catch (SQLException e) { + throw new RuntimeException(e); + } + } + + private void createCustomersTableIfNotExists() { + try (Connection conn = this.getConnection()) { + PreparedStatement pstmt = conn.prepareStatement( + """ + create table if not exists customers ( + id bigint not null, + name varchar not null, + primary key (id) + ) + """ + ); + pstmt.execute(); + } catch (SQLException e) { + throw new RuntimeException(e); + } + } + + private Connection getConnection() { + try { + return DriverManager.getConnection(url, username, password); + } catch (Exception e) { + throw new RuntimeException(e); + } + } +} +``` + +## JUnit 5 lifecycle callbacks + +When testing with Testcontainers, you want to start the required containers +before executing any tests and remove them afterwards. You can use JUnit 5 +`@BeforeAll` and `@AfterAll` lifecycle callback methods for this: + +```java +package com.testcontainers.demo; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertTrue; + +import java.util.List; +import java.util.Optional; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.testcontainers.postgresql.PostgreSQLContainer; + +class CustomerServiceWithLifeCycleCallbacksTest { + + static PostgreSQLContainer postgres = new PostgreSQLContainer( + "postgres:16-alpine" + ); + + CustomerService customerService; + + @BeforeAll + static void startContainers() { + postgres.start(); + } + + @AfterAll + static void stopContainers() { + postgres.stop(); + } + + @BeforeEach + void setUp() { + customerService = + new CustomerService( + postgres.getJdbcUrl(), + postgres.getUsername(), + postgres.getPassword() + ); + customerService.deleteAllCustomers(); + } + + @Test + void shouldCreateCustomer() { + customerService.createCustomer(new Customer(1L, "George")); + + Optional customer = customerService.getCustomer(1L); + assertTrue(customer.isPresent()); + assertEquals(1L, customer.get().id()); + assertEquals("George", customer.get().name()); + } + + @Test + void shouldGetCustomers() { + customerService.createCustomer(new Customer(1L, "George")); + customerService.createCustomer(new Customer(2L, "John")); + + List customers = customerService.getAllCustomers(); + assertEquals(2, customers.size()); + } +} +``` + +Here's what the code does: + +- `PostgreSQLContainer` is declared as a **static field**. The container starts + before all tests and stops after all tests in this class. +- `@BeforeAll` starts the container, `@AfterAll` stops it. +- `@BeforeEach` initializes `CustomerService` with the container's JDBC + parameters and deletes all rows to give each test a clean database. + +Key observations: + +- Because the container is a **static field**, it's shared across all test + methods in the class. You can declare it as a non-static field and use + `@BeforeEach`/`@AfterEach` to start a new container per test, but this + isn't recommended as it's resource-intensive. +- Even without explicitly stopping the container in `@AfterAll`, Testcontainers + uses the [Ryuk container](https://github.com/testcontainers/moby-ryuk) to + clean up containers automatically when the JVM exits. + +## JUnit 5 extension annotations + +The Testcontainers library provides a JUnit 5 extension that simplifies +starting and stopping containers using annotations. To use it, add the +`org.testcontainers:testcontainers-junit-jupiter` test dependency. + +```java +package com.testcontainers.demo; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertTrue; + +import java.util.List; +import java.util.Optional; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.testcontainers.postgresql.PostgreSQLContainer; +import org.testcontainers.junit.jupiter.Container; +import org.testcontainers.junit.jupiter.Testcontainers; + +@Testcontainers +class CustomerServiceWithJUnit5ExtensionTest { + + @Container + static PostgreSQLContainer postgres = new PostgreSQLContainer( + "postgres:16-alpine" + ); + + CustomerService customerService; + + @BeforeEach + void setUp() { + customerService = + new CustomerService( + postgres.getJdbcUrl(), + postgres.getUsername(), + postgres.getPassword() + ); + customerService.deleteAllCustomers(); + } + + @Test + void shouldCreateCustomer() { + customerService.createCustomer(new Customer(1L, "George")); + + Optional customer = customerService.getCustomer(1L); + assertTrue(customer.isPresent()); + assertEquals(1L, customer.get().id()); + assertEquals("George", customer.get().name()); + } + + @Test + void shouldGetCustomers() { + customerService.createCustomer(new Customer(1L, "George")); + customerService.createCustomer(new Customer(2L, "John")); + + List customers = customerService.getAllCustomers(); + assertEquals(2, customers.size()); + } +} +``` + +Instead of manually starting and stopping the container in `@BeforeAll` and +`@AfterAll`, the `@Testcontainers` annotation on the class and the +`@Container` annotation on the field handle it automatically: + +- The extension finds all `@Container`-annotated fields. +- **Static fields** start once before all tests and stop after all tests. +- **Instance fields** start before each test and stop after each test (not + recommended — it's resource-intensive). + +## Singleton containers pattern + +As the number of test classes grows, starting containers for each class adds +up. The singleton containers pattern starts all required containers once in a +common base class and reuses them across all integration tests. + +### Define the base class + +Create an abstract base class that starts the containers in a static +initializer: + +```java +package com.testcontainers.demo; + +import org.testcontainers.postgresql.PostgreSQLContainer; +import org.testcontainers.kafka.ConfluentKafkaContainer; + +public abstract class AbstractIntegrationTest { + + static PostgreSQLContainer postgres = new PostgreSQLContainer( + "postgres:16-alpine"); + static ConfluentKafkaContainer kafka = new ConfluentKafkaContainer( + "confluentinc/cp-kafka:7.8.0"); + + static { + postgres.start(); + kafka.start(); + } +} +``` + +The containers start once when the class loads and Testcontainers uses the +[Ryuk container](https://github.com/testcontainers/moby-ryuk) to remove them +after the JVM exits. + +> [!TIP] +> Instead of starting containers sequentially, start them in parallel using +> `Startables.deepStart(postgres, kafka).join();` + +### Extend the base class + +Each test class inherits from the base class and reuses the same containers: + +```java +class ProductControllerTest extends AbstractIntegrationTest { + + ProductRepository productRepository; + + @BeforeEach + void setUp() { + productRepository = new ProductRepository(...); + productRepository.deleteAll(); + } + + @Test + void shouldGetAllProducts() { + // test logic using the shared postgres container + } +} +``` + +### Avoid a common misconfiguration + +A common mistake is combining singleton containers with the `@Testcontainers` +and `@Container` annotations: + +```java +// DON'T DO THIS — containers will stop after each test class +@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT) +@Testcontainers +public abstract class AbstractIntegrationTest { + + @Container + static PostgreSQLContainer postgres = new PostgreSQLContainer<>( + DockerImageName.parse("postgres:16-alpine")); + + @DynamicPropertySource + static void configureProperties(DynamicPropertyRegistry registry) { + registry.add("spring.datasource.url", postgres::getJdbcUrl); + registry.add("spring.datasource.username", postgres::getUsername); + registry.add("spring.datasource.password", postgres::getPassword); + } +} +``` + +The `@Testcontainers` extension stops containers at the end of **each test +class**. Subsequent test classes reuse the cached Spring context, but the +containers are already stopped — causing connection failures. + +Instead, use a static initializer or `@BeforeAll` to start the containers, +without the `@Testcontainers` and `@Container` annotations. + +### Summary + +- Use **JUnit 5 lifecycle callbacks** (`@BeforeAll`/`@AfterAll`) for + explicit control over container startup and shutdown. +- Use **extension annotations** (`@Testcontainers`/`@Container`) for less + boilerplate in single test classes. +- Use the **singleton containers pattern** (static initializer in a base class) + to share containers across multiple test classes. +- Don't mix singleton containers with `@Testcontainers`/`@Container` + annotations. + +### Further reading + +- [Testcontainers JUnit 5 quickstart](https://java.testcontainers.org/quickstart/junit_5_quickstart/) +- [Testcontainers singleton containers pattern](https://java.testcontainers.org/test_framework_integration/manual_lifecycle_control/#singleton-containers) +- [Testing a Spring Boot REST API with Testcontainers](/guides/testcontainers-java-spring-boot-rest-api/) diff --git a/content/guides/testcontainers-java-lifecycle/create-project.md b/content/guides/testcontainers-java-lifecycle/create-project.md deleted file mode 100644 index 6ca63e724beb..000000000000 --- a/content/guides/testcontainers-java-lifecycle/create-project.md +++ /dev/null @@ -1,167 +0,0 @@ ---- -title: Create the project and business logic -linkTitle: Create the project -description: Set up a Java project with a PostgreSQL-backed customer service for lifecycle testing. -keywords: testcontainers, java, lifecycle, postgresql, junit, project setup -weight: 10 ---- - -## Set up the project - -Create a Java project with Maven and add the required dependencies: - -```xml - - - org.postgresql - postgresql - 42.7.3 - - - ch.qos.logback - logback-classic - 1.5.6 - - - org.junit.jupiter - junit-jupiter - 5.10.2 - test - - - org.testcontainers - testcontainers-junit-jupiter - 2.0.4 - test - - - org.testcontainers - testcontainers-postgresql - 2.0.4 - test - - -``` - -## Create the business logic - -Create a `Customer` record: - -```java -package com.testcontainers.demo; - -public record Customer(Long id, String name) {} -``` - -Create a `CustomerService` class with methods to create, retrieve, and delete -customers: - -```java -package com.testcontainers.demo; - -import java.sql.Connection; -import java.sql.DriverManager; -import java.sql.PreparedStatement; -import java.sql.ResultSet; -import java.sql.SQLException; -import java.util.ArrayList; -import java.util.List; -import java.util.Optional; - -public class CustomerService { - - private final String url; - private final String username; - private final String password; - - public CustomerService(String url, String username, String password) { - this.url = url; - this.username = username; - this.password = password; - createCustomersTableIfNotExists(); - } - - public void createCustomer(Customer customer) { - try (Connection conn = this.getConnection()) { - PreparedStatement pstmt = conn.prepareStatement( - "insert into customers(id,name) values(?,?)" - ); - pstmt.setLong(1, customer.id()); - pstmt.setString(2, customer.name()); - pstmt.execute(); - } catch (SQLException e) { - throw new RuntimeException(e); - } - } - - public List getAllCustomers() { - List customers = new ArrayList<>(); - try (Connection conn = this.getConnection()) { - PreparedStatement pstmt = conn.prepareStatement( - "select id,name from customers" - ); - ResultSet rs = pstmt.executeQuery(); - while (rs.next()) { - long id = rs.getLong("id"); - String name = rs.getString("name"); - customers.add(new Customer(id, name)); - } - } catch (SQLException e) { - throw new RuntimeException(e); - } - return customers; - } - - public Optional getCustomer(Long customerId) { - try (Connection conn = this.getConnection()) { - PreparedStatement pstmt = conn.prepareStatement( - "select id,name from customers where id = ?" - ); - pstmt.setLong(1, customerId); - ResultSet rs = pstmt.executeQuery(); - if (rs.next()) { - long id = rs.getLong("id"); - String name = rs.getString("name"); - return Optional.of(new Customer(id, name)); - } - } catch (SQLException e) { - throw new RuntimeException(e); - } - return Optional.empty(); - } - - public void deleteAllCustomers() { - try (Connection conn = this.getConnection()) { - PreparedStatement pstmt = conn.prepareStatement("delete from customers"); - pstmt.execute(); - } catch (SQLException e) { - throw new RuntimeException(e); - } - } - - private void createCustomersTableIfNotExists() { - try (Connection conn = this.getConnection()) { - PreparedStatement pstmt = conn.prepareStatement( - """ - create table if not exists customers ( - id bigint not null, - name varchar not null, - primary key (id) - ) - """ - ); - pstmt.execute(); - } catch (SQLException e) { - throw new RuntimeException(e); - } - } - - private Connection getConnection() { - try { - return DriverManager.getConnection(url, username, password); - } catch (Exception e) { - throw new RuntimeException(e); - } - } -} -``` diff --git a/content/guides/testcontainers-java-lifecycle/extension-annotations.md b/content/guides/testcontainers-java-lifecycle/extension-annotations.md deleted file mode 100644 index 790965954136..000000000000 --- a/content/guides/testcontainers-java-lifecycle/extension-annotations.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -title: JUnit 5 extension annotations -linkTitle: Extension annotations -description: Manage Testcontainers container lifecycle using @Testcontainers and @Container annotations. -keywords: testcontainers, java, lifecycle, junit, annotations, container management -weight: 30 ---- - -The Testcontainers library provides a JUnit 5 extension that simplifies -starting and stopping containers using annotations. To use it, add the -`org.testcontainers:testcontainers-junit-jupiter` test dependency. - -```java -package com.testcontainers.demo; - -import static org.junit.jupiter.api.Assertions.assertEquals; -import static org.junit.jupiter.api.Assertions.assertTrue; - -import java.util.List; -import java.util.Optional; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.Test; -import org.testcontainers.postgresql.PostgreSQLContainer; -import org.testcontainers.junit.jupiter.Container; -import org.testcontainers.junit.jupiter.Testcontainers; - -@Testcontainers -class CustomerServiceWithJUnit5ExtensionTest { - - @Container - static PostgreSQLContainer postgres = new PostgreSQLContainer( - "postgres:16-alpine" - ); - - CustomerService customerService; - - @BeforeEach - void setUp() { - customerService = - new CustomerService( - postgres.getJdbcUrl(), - postgres.getUsername(), - postgres.getPassword() - ); - customerService.deleteAllCustomers(); - } - - @Test - void shouldCreateCustomer() { - customerService.createCustomer(new Customer(1L, "George")); - - Optional customer = customerService.getCustomer(1L); - assertTrue(customer.isPresent()); - assertEquals(1L, customer.get().id()); - assertEquals("George", customer.get().name()); - } - - @Test - void shouldGetCustomers() { - customerService.createCustomer(new Customer(1L, "George")); - customerService.createCustomer(new Customer(2L, "John")); - - List customers = customerService.getAllCustomers(); - assertEquals(2, customers.size()); - } -} -``` - -Instead of manually starting and stopping the container in `@BeforeAll` and -`@AfterAll`, the `@Testcontainers` annotation on the class and the -`@Container` annotation on the field handle it automatically: - -- The extension finds all `@Container`-annotated fields. -- **Static fields** start once before all tests and stop after all tests. -- **Instance fields** start before each test and stop after each test (not - recommended — it's resource-intensive). diff --git a/content/guides/testcontainers-java-lifecycle/lifecycle-callbacks.md b/content/guides/testcontainers-java-lifecycle/lifecycle-callbacks.md deleted file mode 100644 index 2aa4207c94f4..000000000000 --- a/content/guides/testcontainers-java-lifecycle/lifecycle-callbacks.md +++ /dev/null @@ -1,93 +0,0 @@ ---- -title: JUnit 5 lifecycle callbacks -linkTitle: Lifecycle callbacks -description: Manage Testcontainers container lifecycle using JUnit 5 @BeforeAll and @AfterAll callbacks. -keywords: testcontainers, java, lifecycle, junit, callbacks, beforeall, afterall -weight: 20 ---- - -When testing with Testcontainers, you want to start the required containers -before executing any tests and remove them afterwards. You can use JUnit 5 -`@BeforeAll` and `@AfterAll` lifecycle callback methods for this: - -```java -package com.testcontainers.demo; - -import static org.junit.jupiter.api.Assertions.assertEquals; -import static org.junit.jupiter.api.Assertions.assertTrue; - -import java.util.List; -import java.util.Optional; -import org.junit.jupiter.api.AfterAll; -import org.junit.jupiter.api.BeforeAll; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.Test; -import org.testcontainers.postgresql.PostgreSQLContainer; - -class CustomerServiceWithLifeCycleCallbacksTest { - - static PostgreSQLContainer postgres = new PostgreSQLContainer( - "postgres:16-alpine" - ); - - CustomerService customerService; - - @BeforeAll - static void startContainers() { - postgres.start(); - } - - @AfterAll - static void stopContainers() { - postgres.stop(); - } - - @BeforeEach - void setUp() { - customerService = - new CustomerService( - postgres.getJdbcUrl(), - postgres.getUsername(), - postgres.getPassword() - ); - customerService.deleteAllCustomers(); - } - - @Test - void shouldCreateCustomer() { - customerService.createCustomer(new Customer(1L, "George")); - - Optional customer = customerService.getCustomer(1L); - assertTrue(customer.isPresent()); - assertEquals(1L, customer.get().id()); - assertEquals("George", customer.get().name()); - } - - @Test - void shouldGetCustomers() { - customerService.createCustomer(new Customer(1L, "George")); - customerService.createCustomer(new Customer(2L, "John")); - - List customers = customerService.getAllCustomers(); - assertEquals(2, customers.size()); - } -} -``` - -Here's what the code does: - -- `PostgreSQLContainer` is declared as a **static field**. The container starts - before all tests and stops after all tests in this class. -- `@BeforeAll` starts the container, `@AfterAll` stops it. -- `@BeforeEach` initializes `CustomerService` with the container's JDBC - parameters and deletes all rows to give each test a clean database. - -Key observations: - -- Because the container is a **static field**, it's shared across all test - methods in the class. You can declare it as a non-static field and use - `@BeforeEach`/`@AfterEach` to start a new container per test, but this - isn't recommended as it's resource-intensive. -- Even without explicitly stopping the container in `@AfterAll`, Testcontainers - uses the [Ryuk container](https://github.com/testcontainers/moby-ryuk) to - clean up containers automatically when the JVM exits. diff --git a/content/guides/testcontainers-java-lifecycle/singleton-containers.md b/content/guides/testcontainers-java-lifecycle/singleton-containers.md deleted file mode 100644 index 58f05b3e2e46..000000000000 --- a/content/guides/testcontainers-java-lifecycle/singleton-containers.md +++ /dev/null @@ -1,114 +0,0 @@ ---- -title: Singleton containers pattern -linkTitle: Singleton containers -description: Share containers across multiple test classes using the singleton containers pattern. -keywords: testcontainers, java, lifecycle, singleton, container reuse, junit -weight: 40 ---- - -As the number of test classes grows, starting containers for each class adds -up. The singleton containers pattern starts all required containers once in a -common base class and reuses them across all integration tests. - -## Define the base class - -Create an abstract base class that starts the containers in a static -initializer: - -```java -package com.testcontainers.demo; - -import org.testcontainers.postgresql.PostgreSQLContainer; -import org.testcontainers.kafka.ConfluentKafkaContainer; - -public abstract class AbstractIntegrationTest { - - static PostgreSQLContainer postgres = new PostgreSQLContainer( - "postgres:16-alpine"); - static ConfluentKafkaContainer kafka = new ConfluentKafkaContainer( - "confluentinc/cp-kafka:7.8.0"); - - static { - postgres.start(); - kafka.start(); - } -} -``` - -The containers start once when the class loads and Testcontainers uses the -[Ryuk container](https://github.com/testcontainers/moby-ryuk) to remove them -after the JVM exits. - -> [!TIP] -> Instead of starting containers sequentially, start them in parallel using -> `Startables.deepStart(postgres, kafka).join();` - -## Extend the base class - -Each test class inherits from the base class and reuses the same containers: - -```java -class ProductControllerTest extends AbstractIntegrationTest { - - ProductRepository productRepository; - - @BeforeEach - void setUp() { - productRepository = new ProductRepository(...); - productRepository.deleteAll(); - } - - @Test - void shouldGetAllProducts() { - // test logic using the shared postgres container - } -} -``` - -## Avoid a common misconfiguration - -A common mistake is combining singleton containers with the `@Testcontainers` -and `@Container` annotations: - -```java -// DON'T DO THIS — containers will stop after each test class -@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT) -@Testcontainers -public abstract class AbstractIntegrationTest { - - @Container - static PostgreSQLContainer postgres = new PostgreSQLContainer<>( - DockerImageName.parse("postgres:16-alpine")); - - @DynamicPropertySource - static void configureProperties(DynamicPropertyRegistry registry) { - registry.add("spring.datasource.url", postgres::getJdbcUrl); - registry.add("spring.datasource.username", postgres::getUsername); - registry.add("spring.datasource.password", postgres::getPassword); - } -} -``` - -The `@Testcontainers` extension stops containers at the end of **each test -class**. Subsequent test classes reuse the cached Spring context, but the -containers are already stopped — causing connection failures. - -Instead, use a static initializer or `@BeforeAll` to start the containers, -without the `@Testcontainers` and `@Container` annotations. - -## Summary - -- Use **JUnit 5 lifecycle callbacks** (`@BeforeAll`/`@AfterAll`) for - explicit control over container startup and shutdown. -- Use **extension annotations** (`@Testcontainers`/`@Container`) for less - boilerplate in single test classes. -- Use the **singleton containers pattern** (static initializer in a base class) - to share containers across multiple test classes. -- Don't mix singleton containers with `@Testcontainers`/`@Container` - annotations. - -## Further reading - -- [Testcontainers JUnit 5 quickstart](https://java.testcontainers.org/quickstart/junit_5_quickstart/) -- [Testcontainers singleton containers pattern](https://java.testcontainers.org/test_framework_integration/manual_lifecycle_control/#singleton-containers) -- [Testing a Spring Boot REST API with Testcontainers](/guides/testcontainers-java-spring-boot-rest-api/) diff --git a/content/guides/testcontainers-java-micronaut-kafka/_index.md b/content/guides/testcontainers-java-micronaut-kafka/_index.md index 984349bc9c1f..4aa9e9263765 100644 --- a/content/guides/testcontainers-java-micronaut-kafka/_index.md +++ b/content/guides/testcontainers-java-micronaut-kafka/_index.md @@ -6,14 +6,16 @@ keywords: testcontainers, java, micronaut, testing, kafka, mysql, jpa, awaitilit summary: | Learn how to create a Micronaut application with a Kafka listener that persists data in MySQL, then test it using Testcontainers Kafka and MySQL modules with Awaitility. -toc_min: 1 -toc_max: 2 -tags: [testing-with-docker] -languages: [java] +aliases: + - /guides/testcontainers-java-micronaut-kafka/create-project/ + - /guides/testcontainers-java-micronaut-kafka/run-tests/ + - /guides/testcontainers-java-micronaut-kafka/write-tests/ params: + tags: [testing] time: 25 minutes --- + In this guide, you'll learn how to: @@ -32,3 +34,439 @@ In this guide, you'll learn how to: > If you're new to Testcontainers, visit the > [Testcontainers overview](https://testcontainers.com/getting-started/) to learn more about > Testcontainers and the benefits of using it. + +## Create the Micronaut project + +### Set up the project + +Create a Micronaut project from [Micronaut Launch](https://micronaut.io/launch) by +selecting the **kafka**, **data-jpa**, **mysql**, **awaitility**, **assertj**, and +**testcontainers** features. + +Alternatively, clone the +[guide repository](https://github.com/testcontainers/tc-guide-testing-micronaut-kafka-listener). + +You'll use the [Awaitility](http://www.awaitility.org/) library to assert the +expectations of an asynchronous process flow. + +The key dependencies in `pom.xml` are: + +```xml + + io.micronaut.platform + micronaut-parent + 4.1.4 + + + + io.micronaut.data + micronaut-data-hibernate-jpa + compile + + + io.micronaut.kafka + micronaut-kafka + compile + + + io.micronaut.serde + micronaut-serde-jackson + compile + + + io.micronaut.sql + micronaut-jdbc-hikari + compile + + + mysql + mysql-connector-java + runtime + + + org.awaitility + awaitility + 4.2.0 + test + + + org.testcontainers + testcontainers-junit-jupiter + test + + + org.testcontainers + testcontainers-kafka + test + + + org.testcontainers + testcontainers-mysql + test + + +``` + +The Micronaut parent POM manages the Testcontainers BOM, so you don't need to +specify versions for Testcontainers modules individually. + +### Create the JPA entity + +The application listens to a topic called `product-price-changes`. When a +message arrives, it extracts the product code and price from the event payload +and updates the price for that product in the MySQL database. + +Create `Product.java`: + +```java +package com.testcontainers.demo; + +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.GeneratedValue; +import jakarta.persistence.GenerationType; +import jakarta.persistence.Id; +import jakarta.persistence.Table; +import java.math.BigDecimal; + +@Entity +@Table(name = "products") +public class Product { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(nullable = false, unique = true) + private String code; + + @Column(nullable = false) + private String name; + + @Column(nullable = false) + private BigDecimal price; + + public Product() {} + + public Product(Long id, String code, String name, BigDecimal price) { + this.id = id; + this.code = code; + this.name = name; + this.price = price; + } + + public Long getId() { + return id; + } + + public void setId(Long id) { + this.id = id; + } + + public String getCode() { + return code; + } + + public void setCode(String code) { + this.code = code; + } + + public String getName() { + return name; + } + + public void setName(String name) { + this.name = name; + } + + public BigDecimal getPrice() { + return price; + } + + public void setPrice(BigDecimal price) { + this.price = price; + } +} +``` + +### Create the Micronaut Data JPA repository + +Create a repository interface for the `Product` entity with a method to find a +product by code and a method to update the price for a given product code: + +```java +package com.testcontainers.demo; + +import io.micronaut.data.annotation.Query; +import io.micronaut.data.annotation.Repository; +import io.micronaut.data.jpa.repository.JpaRepository; +import java.math.BigDecimal; +import java.util.Optional; + +@Repository +public interface ProductRepository extends JpaRepository { + + Optional findByCode(String code); + + @Query("update Product p set p.price = :price where p.code = :productCode") + void updateProductPrice(String productCode, BigDecimal price); +} +``` + +Unlike Spring Data JPA, Micronaut Data uses compile-time annotation processing +to implement repository methods, avoiding runtime reflection. + +### Create the event payload + +Create a record named `ProductPriceChangedEvent` that represents the structure +of the event payload received from the Kafka topic: + +```java +package com.testcontainers.demo; + +import io.micronaut.serde.annotation.Serdeable; +import java.math.BigDecimal; + +@Serdeable +public record ProductPriceChangedEvent(String productCode, BigDecimal price) {} +``` + +The `@Serdeable` annotation tells Micronaut Serialization that this type can be +serialized and deserialized. + +The sender and receiver agree on the following JSON format: + +```json +{ + "productCode": "P100", + "price": 25.0 +} +``` + +### Implement the Kafka listener + +Create `ProductPriceChangedEventHandler.java`, which handles messages from the +`product-price-changes` topic and updates the product price in the database: + +```java +package com.testcontainers.demo; + +import static io.micronaut.configuration.kafka.annotation.OffsetReset.EARLIEST; + +import io.micronaut.configuration.kafka.annotation.KafkaListener; +import io.micronaut.configuration.kafka.annotation.Topic; +import jakarta.inject.Singleton; +import jakarta.transaction.Transactional; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +@Singleton +@Transactional +class ProductPriceChangedEventHandler { + + private static final Logger LOG = LoggerFactory.getLogger(ProductPriceChangedEventHandler.class); + + private final ProductRepository productRepository; + + ProductPriceChangedEventHandler(ProductRepository productRepository) { + this.productRepository = productRepository; + } + + @Topic("product-price-changes") + @KafkaListener(offsetReset = EARLIEST, groupId = "demo") + public void handle(ProductPriceChangedEvent event) { + LOG.info("Received a ProductPriceChangedEvent with productCode:{}: ", event.productCode()); + productRepository.updateProductPrice(event.productCode(), event.price()); + } +} +``` + +Key details: + +- The `@KafkaListener` annotation marks this class as a Kafka message listener. + Setting `offsetReset` to `EARLIEST` makes the listener start consuming + messages from the beginning of the partition, which is useful during testing. +- The `@Topic` annotation specifies which topic to subscribe to. +- Micronaut handles JSON deserialization of the `ProductPriceChangedEvent` + automatically using Micronaut Serialization. + +### Configure the datasource + +Add the following properties to `src/main/resources/application.properties`: + +```properties +micronaut.application.name=tc-guide-testing-micronaut-kafka-listener +datasources.default.db-type=mysql +datasources.default.dialect=MYSQL +jpa.default.properties.hibernate.hbm2ddl.auto=update +jpa.default.entity-scan.packages=com.testcontainers.demo +datasources.default.driver-class-name=com.mysql.cj.jdbc.Driver +``` + +Hibernate's `hbm2ddl.auto=update` creates and updates the database schema +automatically. For testing, you'll override this to `create-drop` in the test +properties file. + +Create `src/test/resources/application-test.properties`: + +```properties +jpa.default.properties.hibernate.hbm2ddl.auto=create-drop +``` + +## Write tests with Testcontainers + +To test the Kafka listener, you need a running Kafka broker and a MySQL +database, plus a started Micronaut application context. Testcontainers spins up +both services in Docker containers and the `TestPropertyProvider` interface +connects them to Micronaut. + +### Create a Kafka client for testing + +First, create a `@KafkaClient` interface to publish events in the test: + +```java +package com.testcontainers.demo; + +import io.micronaut.configuration.kafka.annotation.KafkaClient; +import io.micronaut.configuration.kafka.annotation.KafkaKey; +import io.micronaut.configuration.kafka.annotation.Topic; + +@KafkaClient +public interface ProductPriceChangesClient { + + @Topic("product-price-changes") + void send(@KafkaKey String productCode, ProductPriceChangedEvent event); +} +``` + +Key details: + +- The `@KafkaClient` annotation designates this interface as a Kafka producer. +- The `@Topic` annotation specifies the target topic. +- The `@KafkaKey` annotation marks the parameter used as the Kafka message key. + If no such parameter exists, Micronaut sends the record with a null key. + +### Write the test + +Create `ProductPriceChangedEventHandlerTest.java`: + +```java +package com.testcontainers.demo; + +import static java.util.concurrent.TimeUnit.SECONDS; +import static org.assertj.core.api.Assertions.assertThat; +import static org.awaitility.Awaitility.await; + +import io.micronaut.context.annotation.Property; +import io.micronaut.core.annotation.NonNull; +import io.micronaut.test.extensions.junit5.annotation.MicronautTest; +import io.micronaut.test.support.TestPropertyProvider; +import java.math.BigDecimal; +import java.time.Duration; +import java.util.Collections; +import java.util.Map; +import java.util.Optional; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.TestInstance; +import org.testcontainers.kafka.ConfluentKafkaContainer; +import org.testcontainers.junit.jupiter.Container; +import org.testcontainers.junit.jupiter.Testcontainers; + +@MicronautTest(transactional = false) +@Property(name = "datasources.default.driver-class-name", value = "org.testcontainers.jdbc.ContainerDatabaseDriver") +@Property(name = "datasources.default.url", value = "jdbc:tc:mysql:8.0.32:///db") +@Testcontainers(disabledWithoutDocker = true) +@TestInstance(TestInstance.Lifecycle.PER_CLASS) +class ProductPriceChangedEventHandlerTest implements TestPropertyProvider { + + @Container + static final ConfluentKafkaContainer kafka = new ConfluentKafkaContainer("confluentinc/cp-kafka:7.8.0"); + + @Override + public @NonNull Map getProperties() { + if (!kafka.isRunning()) { + kafka.start(); + } + return Collections.singletonMap("kafka.bootstrap.servers", kafka.getBootstrapServers()); + } + + @Test + void shouldHandleProductPriceChangedEvent( + ProductPriceChangesClient productPriceChangesClient, ProductRepository productRepository) { + Product product = new Product(null, "P100", "Product One", BigDecimal.TEN); + Long id = productRepository.save(product).getId(); + + ProductPriceChangedEvent event = new ProductPriceChangedEvent("P100", new BigDecimal("14.50")); + + productPriceChangesClient.send(event.productCode(), event); + + await().pollInterval(Duration.ofSeconds(3)).atMost(10, SECONDS).untilAsserted(() -> { + Optional optionalProduct = productRepository.findByCode("P100"); + assertThat(optionalProduct).isPresent(); + assertThat(optionalProduct.get().getCode()).isEqualTo("P100"); + assertThat(optionalProduct.get().getPrice()).isEqualTo(new BigDecimal("14.50")); + }); + + productRepository.deleteById(id); + } +} +``` + +Here's what the test does: + +- `@MicronautTest` initializes the Micronaut application context and the + embedded server. Setting `transactional` to `false` prevents each test method + from running inside a rolled-back transaction, which is necessary because the + Kafka listener processes messages in a separate thread. +- The `@Property` annotations override the datasource driver and URL to use the + Testcontainers special JDBC URL (`jdbc:tc:mysql:8.0.32:///db`). This spins up + a MySQL container and configures it as the datasource automatically. +- `@Testcontainers` and `@Container` manage the Kafka container lifecycle. + The `TestPropertyProvider` interface registers the Kafka bootstrap servers + with Micronaut so that the producer and consumer connect to the test container. +- `@TestInstance(TestInstance.Lifecycle.PER_CLASS)` creates a single test + instance for all test methods, which is required when implementing + `TestPropertyProvider`. +- The test creates a `Product` record in the database, then sends a + `ProductPriceChangedEvent` to the `product-price-changes` topic using the + `ProductPriceChangesClient`. +- Because Kafka message processing is asynchronous, the test uses + [Awaitility](http://www.awaitility.org/) to poll every 3 seconds (up to a + maximum of 10 seconds) until the product price in the database matches the + expected value. + +## Run tests and next steps + +### Run the tests + +```console +$ ./mvnw test +``` + +Or with Gradle: + +```console +$ ./gradlew test +``` + +You should see the Kafka and MySQL Docker containers start and all tests pass. +After the tests finish, the containers stop and are removed automatically. + +### Summary + +Testing with real Kafka and MySQL instances gives you more confidence in the +correctness of your code than mocks or embedded alternatives. The +Testcontainers library manages the container lifecycle so that your integration +tests run against the same services you use in production. + +To learn more about Testcontainers, visit the +[Testcontainers overview](https://testcontainers.com/getting-started/). + +### Further reading + +- [Testing REST API integrations in Micronaut apps using WireMock](/guides/testcontainers-java-micronaut-wiremock/) +- [Testing Spring Boot Kafka Listener using Testcontainers](/guides/testcontainers-java-spring-boot-kafka/) +- [Getting started with Testcontainers in a Java Spring Boot project](https://testcontainers.com/guides/testing-spring-boot-rest-api-using-testcontainers/) +- [Awaitility](http://www.awaitility.org/) +- [Testcontainers Kafka module](https://java.testcontainers.org/modules/kafka/) +- [Testcontainers MySQL module](https://java.testcontainers.org/modules/databases/mysql/) diff --git a/content/guides/testcontainers-java-micronaut-kafka/create-project.md b/content/guides/testcontainers-java-micronaut-kafka/create-project.md deleted file mode 100644 index bfad0bb92828..000000000000 --- a/content/guides/testcontainers-java-micronaut-kafka/create-project.md +++ /dev/null @@ -1,283 +0,0 @@ ---- -title: Create the Micronaut project -linkTitle: Create the project -description: Set up a Micronaut project with Kafka, Micronaut Data JPA, and MySQL. -keywords: testcontainers, java, micronaut, kafka, mysql, project setup -weight: 10 ---- - -## Set up the project - -Create a Micronaut project from [Micronaut Launch](https://micronaut.io/launch) by -selecting the **kafka**, **data-jpa**, **mysql**, **awaitility**, **assertj**, and -**testcontainers** features. - -Alternatively, clone the -[guide repository](https://github.com/testcontainers/tc-guide-testing-micronaut-kafka-listener). - -You'll use the [Awaitility](http://www.awaitility.org/) library to assert the -expectations of an asynchronous process flow. - -The key dependencies in `pom.xml` are: - -```xml - - io.micronaut.platform - micronaut-parent - 4.1.4 - - - - io.micronaut.data - micronaut-data-hibernate-jpa - compile - - - io.micronaut.kafka - micronaut-kafka - compile - - - io.micronaut.serde - micronaut-serde-jackson - compile - - - io.micronaut.sql - micronaut-jdbc-hikari - compile - - - mysql - mysql-connector-java - runtime - - - org.awaitility - awaitility - 4.2.0 - test - - - org.testcontainers - testcontainers-junit-jupiter - test - - - org.testcontainers - testcontainers-kafka - test - - - org.testcontainers - testcontainers-mysql - test - - -``` - -The Micronaut parent POM manages the Testcontainers BOM, so you don't need to -specify versions for Testcontainers modules individually. - -## Create the JPA entity - -The application listens to a topic called `product-price-changes`. When a -message arrives, it extracts the product code and price from the event payload -and updates the price for that product in the MySQL database. - -Create `Product.java`: - -```java -package com.testcontainers.demo; - -import jakarta.persistence.Column; -import jakarta.persistence.Entity; -import jakarta.persistence.GeneratedValue; -import jakarta.persistence.GenerationType; -import jakarta.persistence.Id; -import jakarta.persistence.Table; -import java.math.BigDecimal; - -@Entity -@Table(name = "products") -public class Product { - - @Id - @GeneratedValue(strategy = GenerationType.IDENTITY) - private Long id; - - @Column(nullable = false, unique = true) - private String code; - - @Column(nullable = false) - private String name; - - @Column(nullable = false) - private BigDecimal price; - - public Product() {} - - public Product(Long id, String code, String name, BigDecimal price) { - this.id = id; - this.code = code; - this.name = name; - this.price = price; - } - - public Long getId() { - return id; - } - - public void setId(Long id) { - this.id = id; - } - - public String getCode() { - return code; - } - - public void setCode(String code) { - this.code = code; - } - - public String getName() { - return name; - } - - public void setName(String name) { - this.name = name; - } - - public BigDecimal getPrice() { - return price; - } - - public void setPrice(BigDecimal price) { - this.price = price; - } -} -``` - -## Create the Micronaut Data JPA repository - -Create a repository interface for the `Product` entity with a method to find a -product by code and a method to update the price for a given product code: - -```java -package com.testcontainers.demo; - -import io.micronaut.data.annotation.Query; -import io.micronaut.data.annotation.Repository; -import io.micronaut.data.jpa.repository.JpaRepository; -import java.math.BigDecimal; -import java.util.Optional; - -@Repository -public interface ProductRepository extends JpaRepository { - - Optional findByCode(String code); - - @Query("update Product p set p.price = :price where p.code = :productCode") - void updateProductPrice(String productCode, BigDecimal price); -} -``` - -Unlike Spring Data JPA, Micronaut Data uses compile-time annotation processing -to implement repository methods, avoiding runtime reflection. - -## Create the event payload - -Create a record named `ProductPriceChangedEvent` that represents the structure -of the event payload received from the Kafka topic: - -```java -package com.testcontainers.demo; - -import io.micronaut.serde.annotation.Serdeable; -import java.math.BigDecimal; - -@Serdeable -public record ProductPriceChangedEvent(String productCode, BigDecimal price) {} -``` - -The `@Serdeable` annotation tells Micronaut Serialization that this type can be -serialized and deserialized. - -The sender and receiver agree on the following JSON format: - -```json -{ - "productCode": "P100", - "price": 25.0 -} -``` - -## Implement the Kafka listener - -Create `ProductPriceChangedEventHandler.java`, which handles messages from the -`product-price-changes` topic and updates the product price in the database: - -```java -package com.testcontainers.demo; - -import static io.micronaut.configuration.kafka.annotation.OffsetReset.EARLIEST; - -import io.micronaut.configuration.kafka.annotation.KafkaListener; -import io.micronaut.configuration.kafka.annotation.Topic; -import jakarta.inject.Singleton; -import jakarta.transaction.Transactional; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -@Singleton -@Transactional -class ProductPriceChangedEventHandler { - - private static final Logger LOG = LoggerFactory.getLogger(ProductPriceChangedEventHandler.class); - - private final ProductRepository productRepository; - - ProductPriceChangedEventHandler(ProductRepository productRepository) { - this.productRepository = productRepository; - } - - @Topic("product-price-changes") - @KafkaListener(offsetReset = EARLIEST, groupId = "demo") - public void handle(ProductPriceChangedEvent event) { - LOG.info("Received a ProductPriceChangedEvent with productCode:{}: ", event.productCode()); - productRepository.updateProductPrice(event.productCode(), event.price()); - } -} -``` - -Key details: - -- The `@KafkaListener` annotation marks this class as a Kafka message listener. - Setting `offsetReset` to `EARLIEST` makes the listener start consuming - messages from the beginning of the partition, which is useful during testing. -- The `@Topic` annotation specifies which topic to subscribe to. -- Micronaut handles JSON deserialization of the `ProductPriceChangedEvent` - automatically using Micronaut Serialization. - -## Configure the datasource - -Add the following properties to `src/main/resources/application.properties`: - -```properties -micronaut.application.name=tc-guide-testing-micronaut-kafka-listener -datasources.default.db-type=mysql -datasources.default.dialect=MYSQL -jpa.default.properties.hibernate.hbm2ddl.auto=update -jpa.default.entity-scan.packages=com.testcontainers.demo -datasources.default.driver-class-name=com.mysql.cj.jdbc.Driver -``` - -Hibernate's `hbm2ddl.auto=update` creates and updates the database schema -automatically. For testing, you'll override this to `create-drop` in the test -properties file. - -Create `src/test/resources/application-test.properties`: - -```properties -jpa.default.properties.hibernate.hbm2ddl.auto=create-drop -``` diff --git a/content/guides/testcontainers-java-micronaut-kafka/run-tests.md b/content/guides/testcontainers-java-micronaut-kafka/run-tests.md deleted file mode 100644 index c2f8fe3db30f..000000000000 --- a/content/guides/testcontainers-java-micronaut-kafka/run-tests.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -title: Run tests and next steps -linkTitle: Run tests -description: Run your Testcontainers-based Micronaut Kafka integration tests and explore next steps. -keywords: testcontainers, java, micronaut, kafka, mysql, integration testing -weight: 30 ---- - -## Run the tests - -```console -$ ./mvnw test -``` - -Or with Gradle: - -```console -$ ./gradlew test -``` - -You should see the Kafka and MySQL Docker containers start and all tests pass. -After the tests finish, the containers stop and are removed automatically. - -## Summary - -Testing with real Kafka and MySQL instances gives you more confidence in the -correctness of your code than mocks or embedded alternatives. The -Testcontainers library manages the container lifecycle so that your integration -tests run against the same services you use in production. - -To learn more about Testcontainers, visit the -[Testcontainers overview](https://testcontainers.com/getting-started/). - -## Further reading - -- [Testing REST API integrations in Micronaut apps using WireMock](/guides/testcontainers-java-micronaut-wiremock/) -- [Testing Spring Boot Kafka Listener using Testcontainers](/guides/testcontainers-java-spring-boot-kafka/) -- [Getting started with Testcontainers in a Java Spring Boot project](https://testcontainers.com/guides/testing-spring-boot-rest-api-using-testcontainers/) -- [Awaitility](http://www.awaitility.org/) -- [Testcontainers Kafka module](https://java.testcontainers.org/modules/kafka/) -- [Testcontainers MySQL module](https://java.testcontainers.org/modules/databases/mysql/) diff --git a/content/guides/testcontainers-java-micronaut-kafka/write-tests.md b/content/guides/testcontainers-java-micronaut-kafka/write-tests.md deleted file mode 100644 index 87bcd02c693e..000000000000 --- a/content/guides/testcontainers-java-micronaut-kafka/write-tests.md +++ /dev/null @@ -1,127 +0,0 @@ ---- -title: Write tests with Testcontainers -linkTitle: Write tests -description: Test the Micronaut Kafka listener using Testcontainers Kafka and MySQL modules with Awaitility. -keywords: testcontainers, java, micronaut, kafka, mysql, awaitility, integration testing -weight: 20 ---- - -To test the Kafka listener, you need a running Kafka broker and a MySQL -database, plus a started Micronaut application context. Testcontainers spins up -both services in Docker containers and the `TestPropertyProvider` interface -connects them to Micronaut. - -## Create a Kafka client for testing - -First, create a `@KafkaClient` interface to publish events in the test: - -```java -package com.testcontainers.demo; - -import io.micronaut.configuration.kafka.annotation.KafkaClient; -import io.micronaut.configuration.kafka.annotation.KafkaKey; -import io.micronaut.configuration.kafka.annotation.Topic; - -@KafkaClient -public interface ProductPriceChangesClient { - - @Topic("product-price-changes") - void send(@KafkaKey String productCode, ProductPriceChangedEvent event); -} -``` - -Key details: - -- The `@KafkaClient` annotation designates this interface as a Kafka producer. -- The `@Topic` annotation specifies the target topic. -- The `@KafkaKey` annotation marks the parameter used as the Kafka message key. - If no such parameter exists, Micronaut sends the record with a null key. - -## Write the test - -Create `ProductPriceChangedEventHandlerTest.java`: - -```java -package com.testcontainers.demo; - -import static java.util.concurrent.TimeUnit.SECONDS; -import static org.assertj.core.api.Assertions.assertThat; -import static org.awaitility.Awaitility.await; - -import io.micronaut.context.annotation.Property; -import io.micronaut.core.annotation.NonNull; -import io.micronaut.test.extensions.junit5.annotation.MicronautTest; -import io.micronaut.test.support.TestPropertyProvider; -import java.math.BigDecimal; -import java.time.Duration; -import java.util.Collections; -import java.util.Map; -import java.util.Optional; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.TestInstance; -import org.testcontainers.kafka.ConfluentKafkaContainer; -import org.testcontainers.junit.jupiter.Container; -import org.testcontainers.junit.jupiter.Testcontainers; - -@MicronautTest(transactional = false) -@Property(name = "datasources.default.driver-class-name", value = "org.testcontainers.jdbc.ContainerDatabaseDriver") -@Property(name = "datasources.default.url", value = "jdbc:tc:mysql:8.0.32:///db") -@Testcontainers(disabledWithoutDocker = true) -@TestInstance(TestInstance.Lifecycle.PER_CLASS) -class ProductPriceChangedEventHandlerTest implements TestPropertyProvider { - - @Container - static final ConfluentKafkaContainer kafka = new ConfluentKafkaContainer("confluentinc/cp-kafka:7.8.0"); - - @Override - public @NonNull Map getProperties() { - if (!kafka.isRunning()) { - kafka.start(); - } - return Collections.singletonMap("kafka.bootstrap.servers", kafka.getBootstrapServers()); - } - - @Test - void shouldHandleProductPriceChangedEvent( - ProductPriceChangesClient productPriceChangesClient, ProductRepository productRepository) { - Product product = new Product(null, "P100", "Product One", BigDecimal.TEN); - Long id = productRepository.save(product).getId(); - - ProductPriceChangedEvent event = new ProductPriceChangedEvent("P100", new BigDecimal("14.50")); - - productPriceChangesClient.send(event.productCode(), event); - - await().pollInterval(Duration.ofSeconds(3)).atMost(10, SECONDS).untilAsserted(() -> { - Optional optionalProduct = productRepository.findByCode("P100"); - assertThat(optionalProduct).isPresent(); - assertThat(optionalProduct.get().getCode()).isEqualTo("P100"); - assertThat(optionalProduct.get().getPrice()).isEqualTo(new BigDecimal("14.50")); - }); - - productRepository.deleteById(id); - } -} -``` - -Here's what the test does: - -- `@MicronautTest` initializes the Micronaut application context and the - embedded server. Setting `transactional` to `false` prevents each test method - from running inside a rolled-back transaction, which is necessary because the - Kafka listener processes messages in a separate thread. -- The `@Property` annotations override the datasource driver and URL to use the - Testcontainers special JDBC URL (`jdbc:tc:mysql:8.0.32:///db`). This spins up - a MySQL container and configures it as the datasource automatically. -- `@Testcontainers` and `@Container` manage the Kafka container lifecycle. - The `TestPropertyProvider` interface registers the Kafka bootstrap servers - with Micronaut so that the producer and consumer connect to the test container. -- `@TestInstance(TestInstance.Lifecycle.PER_CLASS)` creates a single test - instance for all test methods, which is required when implementing - `TestPropertyProvider`. -- The test creates a `Product` record in the database, then sends a - `ProductPriceChangedEvent` to the `product-price-changes` topic using the - `ProductPriceChangesClient`. -- Because Kafka message processing is asynchronous, the test uses - [Awaitility](http://www.awaitility.org/) to poll every 3 seconds (up to a - maximum of 10 seconds) until the product price in the database matches the - expected value. diff --git a/content/guides/testcontainers-java-micronaut-wiremock/_index.md b/content/guides/testcontainers-java-micronaut-wiremock/_index.md index 03923c238572..9cd2b0b05265 100644 --- a/content/guides/testcontainers-java-micronaut-wiremock/_index.md +++ b/content/guides/testcontainers-java-micronaut-wiremock/_index.md @@ -7,14 +7,16 @@ summary: | Learn how to create a Micronaut application that integrates with external REST APIs, then test those integrations using WireMock and the Testcontainers WireMock module. -toc_min: 1 -toc_max: 2 -tags: [testing-with-docker] -languages: [java] +aliases: + - /guides/testcontainers-java-micronaut-wiremock/create-project/ + - /guides/testcontainers-java-micronaut-wiremock/run-tests/ + - /guides/testcontainers-java-micronaut-wiremock/write-tests/ params: + tags: [testing] time: 20 minutes --- + In this guide, you'll learn how to: @@ -33,3 +35,639 @@ In this guide, you'll learn how to: > If you're new to Testcontainers, visit the > [Testcontainers overview](https://testcontainers.com/getting-started/) to learn more about > Testcontainers and the benefits of using it. + +## Create the Micronaut project + +### Set up the project + +Create a Micronaut project from [Micronaut Launch](https://micronaut.io/launch) +by selecting the **http-client**, **micronaut-test-rest-assured**, and +**testcontainers** features. + +Alternatively, clone the +[guide repository](https://github.com/testcontainers/tc-guide-testing-rest-api-integrations-in-micronaut-apps-using-wiremock). + +After generating the project, add the **WireMock** and **Testcontainers +WireMock** libraries as test dependencies. The key dependencies in `pom.xml` +are: + +```xml + + io.micronaut.platform + micronaut-parent + 4.1.2 + + + + 17 + 4.1.2 + netty + + + + + jitpack.io + https://jitpack.io + + + + + + io.micronaut + micronaut-http-client + compile + + + io.micronaut + micronaut-http-server-netty + compile + + + io.micronaut.serde + micronaut-serde-jackson + compile + + + io.micronaut.test + micronaut-test-junit5 + test + + + io.micronaut.test + micronaut-test-rest-assured + test + + + org.testcontainers + testcontainers-junit-jupiter + test + + + org.testcontainers + testcontainers + test + + + org.wiremock + wiremock-standalone + 3.2.0 + test + + + org.wiremock.integrations.testcontainers + wiremock-testcontainers-module + 1.0-alpha-13 + test + + +``` + +This guide builds an application that manages video albums. A third-party REST +API handles photo assets. For demonstration purposes, the application uses the +publicly available [JSONPlaceholder](https://jsonplaceholder.typicode.com/) API +as a photo service. + +The application exposes a `GET /api/albums/{albumId}` endpoint that calls the +photo service to fetch photos for a given album. +[WireMock](https://wiremock.org/) is a tool for building mock APIs. +Testcontainers provides a +[WireMock module](https://testcontainers.com/modules/wiremock/) that runs +WireMock as a Docker container. + +### Create the Album and Photo models + +Create `Album.java` using Java records. Annotate both records with `@Serdeable` +to allow serialization and deserialization: + +```java +package com.testcontainers.demo; + +import io.micronaut.serde.annotation.Serdeable; +import java.util.List; + +@Serdeable +public record Album(Long albumId, List photos) {} + +@Serdeable +record Photo(Long id, String title, String url, String thumbnailUrl) {} +``` + +### Create the PhotoServiceClient + +Micronaut provides +[declarative HTTP client](https://docs.micronaut.io/latest/guide/#httpClient) +support. Create an interface with a method that fetches photos for a given album +ID: + +```java +package com.testcontainers.demo; + +import io.micronaut.http.annotation.Get; +import io.micronaut.http.annotation.PathVariable; +import io.micronaut.http.client.annotation.Client; +import java.util.List; + +@Client(id = "photosapi") +interface PhotoServiceClient { + + @Get("/albums/{albumId}/photos") + List getPhotos(@PathVariable Long albumId); +} +``` + +The `@Client(id = "photosapi")` annotation ties this client to a named +configuration. Add the following property to +`src/main/resources/application.properties` to set the base URL: + +```properties +micronaut.http.services.photosapi.url=https://jsonplaceholder.typicode.com +``` + +### Create the REST API endpoint + +Create `AlbumController.java`: + +```java +package com.testcontainers.demo; + +import static io.micronaut.scheduling.TaskExecutors.BLOCKING; + +import io.micronaut.http.annotation.Controller; +import io.micronaut.http.annotation.Get; +import io.micronaut.http.annotation.PathVariable; +import io.micronaut.scheduling.annotation.ExecuteOn; + +@Controller("/api") +class AlbumController { + + private final PhotoServiceClient photoServiceClient; + + AlbumController(PhotoServiceClient photoServiceClient) { + this.photoServiceClient = photoServiceClient; + } + + @ExecuteOn(BLOCKING) + @Get("/albums/{albumId}") + public Album getAlbumById(@PathVariable Long albumId) { + return new Album(albumId, photoServiceClient.getPhotos(albumId)); + } +} +``` + +Here's what this controller does: + +- `@Controller("/api")` maps the controller to the `/api` path. +- Constructor injection provides a `PhotoServiceClient` bean. +- `@ExecuteOn(BLOCKING)` offloads blocking I/O to a separate thread pool so it + doesn't block the event loop. +- `@Get("/albums/{albumId}")` maps the `getAlbumById()` method to an HTTP GET + request. + +This endpoint calls the photo service for a given album ID and returns a +response like: + +```json +{ + "albumId": 1, + "photos": [ + { + "id": 51, + "title": "non sunt voluptatem placeat consequuntur rem incidunt", + "url": "https://via.placeholder.com/600/8e973b", + "thumbnailUrl": "https://via.placeholder.com/150/8e973b" + }, + { + "id": 52, + "title": "eveniet pariatur quia nobis reiciendis laboriosam ea", + "url": "https://via.placeholder.com/600/121fa4", + "thumbnailUrl": "https://via.placeholder.com/150/121fa4" + } + ] +} +``` + +## Write tests with WireMock and Testcontainers + +Mocking external API interactions at the HTTP protocol level, rather than +mocking Java methods, lets you verify marshalling and unmarshalling behavior and +simulate network issues. + +### Test with WireMock's JUnit 5 extension + +The first approach uses WireMock's `WireMockExtension` to start an in-process +WireMock server on a dynamic port. + +Create `AlbumControllerTest.java`: + +```java +package com.testcontainers.demo; + +import static com.github.tomakehurst.wiremock.client.WireMock.aResponse; +import static com.github.tomakehurst.wiremock.client.WireMock.urlMatching; +import static com.github.tomakehurst.wiremock.core.WireMockConfiguration.wireMockConfig; +import static io.restassured.RestAssured.given; +import static org.hamcrest.CoreMatchers.is; +import static org.hamcrest.Matchers.hasSize; + +import com.github.tomakehurst.wiremock.client.WireMock; +import com.github.tomakehurst.wiremock.junit5.WireMockExtension; +import io.micronaut.context.ApplicationContext; +import io.micronaut.http.MediaType; +import io.micronaut.runtime.server.EmbeddedServer; +import io.restassured.RestAssured; +import io.restassured.http.ContentType; +import java.util.Collections; +import java.util.Map; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.RegisterExtension; + +class AlbumControllerTest { + + @RegisterExtension + static WireMockExtension wireMock = WireMockExtension.newInstance() + .options(wireMockConfig().dynamicPort()) + .build(); + + private Map getProperties() { + return Collections.singletonMap("micronaut.http.services.photosapi.url", wireMock.baseUrl()); + } + + @Test + void shouldGetAlbumById() { + try (EmbeddedServer server = ApplicationContext.run(EmbeddedServer.class, getProperties())) { + RestAssured.port = server.getPort(); + Long albumId = 1L; + String responseJson = + """ + [ + { + "id": 1, + "title": "accusamus beatae ad facilis cum similique qui sunt", + "url": "https://via.placeholder.com/600/92c952", + "thumbnailUrl": "https://via.placeholder.com/150/92c952" + }, + { + "id": 2, + "title": "reprehenderit est deserunt velit ipsam", + "url": "https://via.placeholder.com/600/771796", + "thumbnailUrl": "https://via.placeholder.com/150/771796" + } + ] + """; + wireMock.stubFor(WireMock.get(urlMatching("/albums/" + albumId + "/photos")) + .willReturn(aResponse() + .withHeader("Content-Type", MediaType.APPLICATION_JSON) + .withBody(responseJson))); + + given().contentType(ContentType.JSON) + .when() + .get("/api/albums/{albumId}", albumId) + .then() + .statusCode(200) + .body("albumId", is(albumId.intValue())) + .body("photos", hasSize(2)); + } + } + + @Test + void shouldReturnServerErrorWhenPhotoServiceCallFailed() { + try (EmbeddedServer server = ApplicationContext.run(EmbeddedServer.class, getProperties())) { + RestAssured.port = server.getPort(); + Long albumId = 2L; + wireMock.stubFor(WireMock.get(urlMatching("/albums/" + albumId + "/photos")) + .willReturn(aResponse().withStatus(500))); + + given().contentType(ContentType.JSON) + .when() + .get("/api/albums/{albumId}", albumId) + .then() + .statusCode(500); + } + } +} +``` + +Here's what this test does: + +- `WireMockExtension` starts a WireMock server on a dynamic port. +- The `getProperties()` method overrides `micronaut.http.services.photosapi.url` + to point at the WireMock endpoint, so the application talks to WireMock + instead of the real photo service. +- `shouldGetAlbumById()` configures a mock response for + `/albums/{albumId}/photos`, sends a request to the application's + `/api/albums/{albumId}` endpoint, and verifies the response body. +- `shouldReturnServerErrorWhenPhotoServiceCallFailed()` configures WireMock to + return a 500 status and verifies the application propagates that error. + +### Stub using JSON mapping files + +Instead of stubbing with the WireMock Java API, you can use JSON mapping-based +configuration. + +Create `src/test/resources/wiremock/mappings/get-album-photos.json`: + +```json +{ + "mappings": [ + { + "request": { + "method": "GET", + "urlPattern": "/albums/([0-9]+)/photos" + }, + "response": { + "status": 200, + "headers": { + "Content-Type": "application/json" + }, + "bodyFileName": "album-photos-resp-200.json" + } + }, + { + "request": { + "method": "GET", + "urlPattern": "/albums/2/photos" + }, + "response": { + "status": 500, + "headers": { + "Content-Type": "application/json" + } + } + }, + { + "request": { + "method": "GET", + "urlPattern": "/albums/3/photos" + }, + "response": { + "status": 200, + "headers": { + "Content-Type": "application/json" + }, + "jsonBody": [] + } + } + ] +} +``` + +Create `src/test/resources/wiremock/__files/album-photos-resp-200.json`: + +```json +[ + { + "id": 1, + "title": "accusamus beatae ad facilis cum similique qui sunt", + "url": "https://via.placeholder.com/600/92c952", + "thumbnailUrl": "https://via.placeholder.com/150/92c952" + }, + { + "id": 2, + "title": "reprehenderit est deserunt velit ipsam", + "url": "https://via.placeholder.com/600/771796", + "thumbnailUrl": "https://via.placeholder.com/150/771796" + } +] +``` + +Then initialize WireMock to load stub mappings from these files: + +```java +@RegisterExtension +static WireMockExtension wireMock = WireMockExtension.newInstance() + .options( + wireMockConfig() + .dynamicPort() + .usingFilesUnderClasspath("wiremock") + ) + .build(); +``` + +With mapping files-based stubbing in place, write tests without needing +programmatic stubs: + +```java +@Test +void shouldGetAlbumById() { + Long albumId = 1L; + try (EmbeddedServer server = ApplicationContext.run(EmbeddedServer.class, getProperties())) { + RestAssured.port = server.getPort(); + + given().contentType(ContentType.JSON) + .when() + .get("/api/albums/{albumId}", albumId) + .then() + .statusCode(200) + .body("albumId", is(albumId.intValue())) + .body("photos", hasSize(2)); + } +} +``` + +### Use the Testcontainers WireMock module + +The [Testcontainers WireMock module](https://testcontainers.com/modules/wiremock/) +provisions a WireMock server as a standalone container within your tests, based +on [WireMock Docker](https://github.com/wiremock/wiremock-docker). + +Create `src/test/resources/mocks-config.json` with the stub mappings: + +```json +{ + "mappings": [ + { + "request": { + "method": "GET", + "urlPattern": "/albums/([0-9]+)/photos" + }, + "response": { + "status": 200, + "headers": { + "Content-Type": "application/json" + }, + "bodyFileName": "album-photos-response.json" + } + }, + { + "request": { + "method": "GET", + "urlPattern": "/albums/2/photos" + }, + "response": { + "status": 500, + "headers": { + "Content-Type": "application/json" + } + } + }, + { + "request": { + "method": "GET", + "urlPattern": "/albums/3/photos" + }, + "response": { + "status": 200, + "headers": { + "Content-Type": "application/json" + }, + "jsonBody": [] + } + } + ] +} +``` + +Create `src/test/resources/album-photos-response.json`: + +```json +[ + { + "id": 1, + "title": "accusamus beatae ad facilis cum similique qui sunt", + "url": "https://via.placeholder.com/600/92c952", + "thumbnailUrl": "https://via.placeholder.com/150/92c952" + }, + { + "id": 2, + "title": "reprehenderit est deserunt velit ipsam", + "url": "https://via.placeholder.com/600/771796", + "thumbnailUrl": "https://via.placeholder.com/150/771796" + } +] +``` + +Create `AlbumControllerTestcontainersTests.java`: + +```java +package com.testcontainers.demo; + +import static io.restassured.RestAssured.given; +import static org.hamcrest.CoreMatchers.is; +import static org.hamcrest.Matchers.hasSize; +import static org.hamcrest.Matchers.nullValue; + +import io.micronaut.context.ApplicationContext; +import io.micronaut.core.annotation.NonNull; +import io.micronaut.runtime.server.EmbeddedServer; +import io.restassured.RestAssured; +import io.restassured.http.ContentType; +import java.util.Collections; +import java.util.Map; +import org.junit.jupiter.api.Test; +import org.testcontainers.junit.jupiter.Container; +import org.testcontainers.junit.jupiter.Testcontainers; +import org.wiremock.integrations.testcontainers.WireMockContainer; + +@Testcontainers(disabledWithoutDocker = true) +class AlbumControllerTestcontainersTests { + + @Container + static WireMockContainer wiremockServer = new WireMockContainer("wiremock/wiremock:2.35.0") + .withMappingFromResource("mocks-config.json") + .withFileFromResource("album-photos-response.json"); + + @NonNull public Map getProperties() { + return Collections.singletonMap("micronaut.http.services.photosapi.url", wiremockServer.getBaseUrl()); + } + + @Test + void shouldGetAlbumById() { + Long albumId = 1L; + try (EmbeddedServer server = ApplicationContext.run(EmbeddedServer.class, getProperties())) { + RestAssured.port = server.getPort(); + + given().contentType(ContentType.JSON) + .when() + .get("/api/albums/{albumId}", albumId) + .then() + .statusCode(200) + .body("albumId", is(albumId.intValue())) + .body("photos", hasSize(2)); + } + } + + @Test + void shouldReturnServerErrorWhenPhotoServiceCallFailed() { + Long albumId = 2L; + try (EmbeddedServer server = ApplicationContext.run(EmbeddedServer.class, getProperties())) { + RestAssured.port = server.getPort(); + given().contentType(ContentType.JSON) + .when() + .get("/api/albums/{albumId}", albumId) + .then() + .statusCode(500); + } + } + + @Test + void shouldReturnEmptyPhotos() { + Long albumId = 3L; + try (EmbeddedServer server = ApplicationContext.run(EmbeddedServer.class, getProperties())) { + RestAssured.port = server.getPort(); + given().contentType(ContentType.JSON) + .when() + .get("/api/albums/{albumId}", albumId) + .then() + .statusCode(200) + .body("albumId", is(albumId.intValue())) + .body("photos", nullValue()); + } + } +} +``` + +Here's what this test does: + +- `@Testcontainers` and `@Container` annotations start a `WireMockContainer` + using the `wiremock/wiremock:2.35.0` Docker image. +- `withMappingFromResource("mocks-config.json")` loads stub mappings from the + classpath resource. +- `withFileFromResource("album-photos-response.json")` makes the response body + file available to WireMock. +- `getProperties()` overrides the photo service URL to point at the WireMock + container's base URL. +- `shouldGetAlbumById()` verifies that the application returns the expected + album with two photos. +- `shouldReturnServerErrorWhenPhotoServiceCallFailed()` verifies that a 500 + from the photo service propagates to the caller. +- `shouldReturnEmptyPhotos()` verifies the application handles an empty photo + list. + +## Run tests and next steps + +### Run the tests + +```console +$ ./mvnw test +``` + +Or with Gradle: + +```console +$ ./gradlew test +``` + +You should see the WireMock Docker container start in the console output. It +acts as the photo service, serving mock responses based on the configured +expectations. All tests should pass. + +### Summary + +You built a Micronaut application that integrates with an external REST API +using declarative HTTP clients, then tested that integration using WireMock and +the Testcontainers WireMock module. Testing at the HTTP protocol level instead +of mocking Java methods lets you catch serialization issues and simulate +realistic failure scenarios. + +> [!TIP] +> Testcontainers WireMock modules are available for Go and Python as well. + +To learn more about Testcontainers, visit the +[Testcontainers overview](https://testcontainers.com/getting-started/). + +### Further reading + +- [Testcontainers WireMock module](https://testcontainers.com/modules/wiremock/) +- [WireMock documentation](https://wiremock.org/docs/) +- [Testcontainers JUnit 5 quickstart](https://java.testcontainers.org/quickstart/junit_5_quickstart/) +- [Testing REST API integrations in Spring Boot using WireMock](/guides/testcontainers-java-wiremock/) diff --git a/content/guides/testcontainers-java-micronaut-wiremock/create-project.md b/content/guides/testcontainers-java-micronaut-wiremock/create-project.md deleted file mode 100644 index 8b4535858e30..000000000000 --- a/content/guides/testcontainers-java-micronaut-wiremock/create-project.md +++ /dev/null @@ -1,215 +0,0 @@ ---- -title: Create the Micronaut project -linkTitle: Create the project -description: Set up a Micronaut project with an external REST API integration using declarative HTTP clients. -keywords: testcontainers, java, micronaut, wiremock, rest api, project setup -weight: 10 ---- - -## Set up the project - -Create a Micronaut project from [Micronaut Launch](https://micronaut.io/launch) -by selecting the **http-client**, **micronaut-test-rest-assured**, and -**testcontainers** features. - -Alternatively, clone the -[guide repository](https://github.com/testcontainers/tc-guide-testing-rest-api-integrations-in-micronaut-apps-using-wiremock). - -After generating the project, add the **WireMock** and **Testcontainers -WireMock** libraries as test dependencies. The key dependencies in `pom.xml` -are: - -```xml - - io.micronaut.platform - micronaut-parent - 4.1.2 - - - - 17 - 4.1.2 - netty - - - - - jitpack.io - https://jitpack.io - - - - - - io.micronaut - micronaut-http-client - compile - - - io.micronaut - micronaut-http-server-netty - compile - - - io.micronaut.serde - micronaut-serde-jackson - compile - - - io.micronaut.test - micronaut-test-junit5 - test - - - io.micronaut.test - micronaut-test-rest-assured - test - - - org.testcontainers - testcontainers-junit-jupiter - test - - - org.testcontainers - testcontainers - test - - - org.wiremock - wiremock-standalone - 3.2.0 - test - - - org.wiremock.integrations.testcontainers - wiremock-testcontainers-module - 1.0-alpha-13 - test - - -``` - -This guide builds an application that manages video albums. A third-party REST -API handles photo assets. For demonstration purposes, the application uses the -publicly available [JSONPlaceholder](https://jsonplaceholder.typicode.com/) API -as a photo service. - -The application exposes a `GET /api/albums/{albumId}` endpoint that calls the -photo service to fetch photos for a given album. -[WireMock](https://wiremock.org/) is a tool for building mock APIs. -Testcontainers provides a -[WireMock module](https://testcontainers.com/modules/wiremock/) that runs -WireMock as a Docker container. - -## Create the Album and Photo models - -Create `Album.java` using Java records. Annotate both records with `@Serdeable` -to allow serialization and deserialization: - -```java -package com.testcontainers.demo; - -import io.micronaut.serde.annotation.Serdeable; -import java.util.List; - -@Serdeable -public record Album(Long albumId, List photos) {} - -@Serdeable -record Photo(Long id, String title, String url, String thumbnailUrl) {} -``` - -## Create the PhotoServiceClient - -Micronaut provides -[declarative HTTP client](https://docs.micronaut.io/latest/guide/#httpClient) -support. Create an interface with a method that fetches photos for a given album -ID: - -```java -package com.testcontainers.demo; - -import io.micronaut.http.annotation.Get; -import io.micronaut.http.annotation.PathVariable; -import io.micronaut.http.client.annotation.Client; -import java.util.List; - -@Client(id = "photosapi") -interface PhotoServiceClient { - - @Get("/albums/{albumId}/photos") - List getPhotos(@PathVariable Long albumId); -} -``` - -The `@Client(id = "photosapi")` annotation ties this client to a named -configuration. Add the following property to -`src/main/resources/application.properties` to set the base URL: - -```properties -micronaut.http.services.photosapi.url=https://jsonplaceholder.typicode.com -``` - -## Create the REST API endpoint - -Create `AlbumController.java`: - -```java -package com.testcontainers.demo; - -import static io.micronaut.scheduling.TaskExecutors.BLOCKING; - -import io.micronaut.http.annotation.Controller; -import io.micronaut.http.annotation.Get; -import io.micronaut.http.annotation.PathVariable; -import io.micronaut.scheduling.annotation.ExecuteOn; - -@Controller("/api") -class AlbumController { - - private final PhotoServiceClient photoServiceClient; - - AlbumController(PhotoServiceClient photoServiceClient) { - this.photoServiceClient = photoServiceClient; - } - - @ExecuteOn(BLOCKING) - @Get("/albums/{albumId}") - public Album getAlbumById(@PathVariable Long albumId) { - return new Album(albumId, photoServiceClient.getPhotos(albumId)); - } -} -``` - -Here's what this controller does: - -- `@Controller("/api")` maps the controller to the `/api` path. -- Constructor injection provides a `PhotoServiceClient` bean. -- `@ExecuteOn(BLOCKING)` offloads blocking I/O to a separate thread pool so it - doesn't block the event loop. -- `@Get("/albums/{albumId}")` maps the `getAlbumById()` method to an HTTP GET - request. - -This endpoint calls the photo service for a given album ID and returns a -response like: - -```json -{ - "albumId": 1, - "photos": [ - { - "id": 51, - "title": "non sunt voluptatem placeat consequuntur rem incidunt", - "url": "https://via.placeholder.com/600/8e973b", - "thumbnailUrl": "https://via.placeholder.com/150/8e973b" - }, - { - "id": 52, - "title": "eveniet pariatur quia nobis reiciendis laboriosam ea", - "url": "https://via.placeholder.com/600/121fa4", - "thumbnailUrl": "https://via.placeholder.com/150/121fa4" - } - ] -} -``` diff --git a/content/guides/testcontainers-java-micronaut-wiremock/run-tests.md b/content/guides/testcontainers-java-micronaut-wiremock/run-tests.md deleted file mode 100644 index a59c6720f779..000000000000 --- a/content/guides/testcontainers-java-micronaut-wiremock/run-tests.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -title: Run tests and next steps -linkTitle: Run tests -description: Run your Testcontainers WireMock integration tests and explore next steps. -keywords: testcontainers, java, micronaut, wiremock, integration testing -weight: 30 ---- - -## Run the tests - -```console -$ ./mvnw test -``` - -Or with Gradle: - -```console -$ ./gradlew test -``` - -You should see the WireMock Docker container start in the console output. It -acts as the photo service, serving mock responses based on the configured -expectations. All tests should pass. - -## Summary - -You built a Micronaut application that integrates with an external REST API -using declarative HTTP clients, then tested that integration using WireMock and -the Testcontainers WireMock module. Testing at the HTTP protocol level instead -of mocking Java methods lets you catch serialization issues and simulate -realistic failure scenarios. - -> [!TIP] -> Testcontainers WireMock modules are available for Go and Python as well. - -To learn more about Testcontainers, visit the -[Testcontainers overview](https://testcontainers.com/getting-started/). - -## Further reading - -- [Testcontainers WireMock module](https://testcontainers.com/modules/wiremock/) -- [WireMock documentation](https://wiremock.org/docs/) -- [Testcontainers JUnit 5 quickstart](https://java.testcontainers.org/quickstart/junit_5_quickstart/) -- [Testing REST API integrations in Spring Boot using WireMock](/guides/testcontainers-java-wiremock/) diff --git a/content/guides/testcontainers-java-micronaut-wiremock/write-tests.md b/content/guides/testcontainers-java-micronaut-wiremock/write-tests.md deleted file mode 100644 index 36b9f9ddfe70..000000000000 --- a/content/guides/testcontainers-java-micronaut-wiremock/write-tests.md +++ /dev/null @@ -1,392 +0,0 @@ ---- -title: Write tests with WireMock and Testcontainers -linkTitle: Write tests -description: Test external REST API integrations using WireMock and the Testcontainers WireMock module. -keywords: testcontainers, java, micronaut, wiremock, rest api, integration testing -weight: 20 ---- - -Mocking external API interactions at the HTTP protocol level, rather than -mocking Java methods, lets you verify marshalling and unmarshalling behavior and -simulate network issues. - -## Test with WireMock's JUnit 5 extension - -The first approach uses WireMock's `WireMockExtension` to start an in-process -WireMock server on a dynamic port. - -Create `AlbumControllerTest.java`: - -```java -package com.testcontainers.demo; - -import static com.github.tomakehurst.wiremock.client.WireMock.aResponse; -import static com.github.tomakehurst.wiremock.client.WireMock.urlMatching; -import static com.github.tomakehurst.wiremock.core.WireMockConfiguration.wireMockConfig; -import static io.restassured.RestAssured.given; -import static org.hamcrest.CoreMatchers.is; -import static org.hamcrest.Matchers.hasSize; - -import com.github.tomakehurst.wiremock.client.WireMock; -import com.github.tomakehurst.wiremock.junit5.WireMockExtension; -import io.micronaut.context.ApplicationContext; -import io.micronaut.http.MediaType; -import io.micronaut.runtime.server.EmbeddedServer; -import io.restassured.RestAssured; -import io.restassured.http.ContentType; -import java.util.Collections; -import java.util.Map; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.extension.RegisterExtension; - -class AlbumControllerTest { - - @RegisterExtension - static WireMockExtension wireMock = WireMockExtension.newInstance() - .options(wireMockConfig().dynamicPort()) - .build(); - - private Map getProperties() { - return Collections.singletonMap("micronaut.http.services.photosapi.url", wireMock.baseUrl()); - } - - @Test - void shouldGetAlbumById() { - try (EmbeddedServer server = ApplicationContext.run(EmbeddedServer.class, getProperties())) { - RestAssured.port = server.getPort(); - Long albumId = 1L; - String responseJson = - """ - [ - { - "id": 1, - "title": "accusamus beatae ad facilis cum similique qui sunt", - "url": "https://via.placeholder.com/600/92c952", - "thumbnailUrl": "https://via.placeholder.com/150/92c952" - }, - { - "id": 2, - "title": "reprehenderit est deserunt velit ipsam", - "url": "https://via.placeholder.com/600/771796", - "thumbnailUrl": "https://via.placeholder.com/150/771796" - } - ] - """; - wireMock.stubFor(WireMock.get(urlMatching("/albums/" + albumId + "/photos")) - .willReturn(aResponse() - .withHeader("Content-Type", MediaType.APPLICATION_JSON) - .withBody(responseJson))); - - given().contentType(ContentType.JSON) - .when() - .get("/api/albums/{albumId}", albumId) - .then() - .statusCode(200) - .body("albumId", is(albumId.intValue())) - .body("photos", hasSize(2)); - } - } - - @Test - void shouldReturnServerErrorWhenPhotoServiceCallFailed() { - try (EmbeddedServer server = ApplicationContext.run(EmbeddedServer.class, getProperties())) { - RestAssured.port = server.getPort(); - Long albumId = 2L; - wireMock.stubFor(WireMock.get(urlMatching("/albums/" + albumId + "/photos")) - .willReturn(aResponse().withStatus(500))); - - given().contentType(ContentType.JSON) - .when() - .get("/api/albums/{albumId}", albumId) - .then() - .statusCode(500); - } - } -} -``` - -Here's what this test does: - -- `WireMockExtension` starts a WireMock server on a dynamic port. -- The `getProperties()` method overrides `micronaut.http.services.photosapi.url` - to point at the WireMock endpoint, so the application talks to WireMock - instead of the real photo service. -- `shouldGetAlbumById()` configures a mock response for - `/albums/{albumId}/photos`, sends a request to the application's - `/api/albums/{albumId}` endpoint, and verifies the response body. -- `shouldReturnServerErrorWhenPhotoServiceCallFailed()` configures WireMock to - return a 500 status and verifies the application propagates that error. - -## Stub using JSON mapping files - -Instead of stubbing with the WireMock Java API, you can use JSON mapping-based -configuration. - -Create `src/test/resources/wiremock/mappings/get-album-photos.json`: - -```json -{ - "mappings": [ - { - "request": { - "method": "GET", - "urlPattern": "/albums/([0-9]+)/photos" - }, - "response": { - "status": 200, - "headers": { - "Content-Type": "application/json" - }, - "bodyFileName": "album-photos-resp-200.json" - } - }, - { - "request": { - "method": "GET", - "urlPattern": "/albums/2/photos" - }, - "response": { - "status": 500, - "headers": { - "Content-Type": "application/json" - } - } - }, - { - "request": { - "method": "GET", - "urlPattern": "/albums/3/photos" - }, - "response": { - "status": 200, - "headers": { - "Content-Type": "application/json" - }, - "jsonBody": [] - } - } - ] -} -``` - -Create `src/test/resources/wiremock/__files/album-photos-resp-200.json`: - -```json -[ - { - "id": 1, - "title": "accusamus beatae ad facilis cum similique qui sunt", - "url": "https://via.placeholder.com/600/92c952", - "thumbnailUrl": "https://via.placeholder.com/150/92c952" - }, - { - "id": 2, - "title": "reprehenderit est deserunt velit ipsam", - "url": "https://via.placeholder.com/600/771796", - "thumbnailUrl": "https://via.placeholder.com/150/771796" - } -] -``` - -Then initialize WireMock to load stub mappings from these files: - -```java -@RegisterExtension -static WireMockExtension wireMock = WireMockExtension.newInstance() - .options( - wireMockConfig() - .dynamicPort() - .usingFilesUnderClasspath("wiremock") - ) - .build(); -``` - -With mapping files-based stubbing in place, write tests without needing -programmatic stubs: - -```java -@Test -void shouldGetAlbumById() { - Long albumId = 1L; - try (EmbeddedServer server = ApplicationContext.run(EmbeddedServer.class, getProperties())) { - RestAssured.port = server.getPort(); - - given().contentType(ContentType.JSON) - .when() - .get("/api/albums/{albumId}", albumId) - .then() - .statusCode(200) - .body("albumId", is(albumId.intValue())) - .body("photos", hasSize(2)); - } -} -``` - -## Use the Testcontainers WireMock module - -The [Testcontainers WireMock module](https://testcontainers.com/modules/wiremock/) -provisions a WireMock server as a standalone container within your tests, based -on [WireMock Docker](https://github.com/wiremock/wiremock-docker). - -Create `src/test/resources/mocks-config.json` with the stub mappings: - -```json -{ - "mappings": [ - { - "request": { - "method": "GET", - "urlPattern": "/albums/([0-9]+)/photos" - }, - "response": { - "status": 200, - "headers": { - "Content-Type": "application/json" - }, - "bodyFileName": "album-photos-response.json" - } - }, - { - "request": { - "method": "GET", - "urlPattern": "/albums/2/photos" - }, - "response": { - "status": 500, - "headers": { - "Content-Type": "application/json" - } - } - }, - { - "request": { - "method": "GET", - "urlPattern": "/albums/3/photos" - }, - "response": { - "status": 200, - "headers": { - "Content-Type": "application/json" - }, - "jsonBody": [] - } - } - ] -} -``` - -Create `src/test/resources/album-photos-response.json`: - -```json -[ - { - "id": 1, - "title": "accusamus beatae ad facilis cum similique qui sunt", - "url": "https://via.placeholder.com/600/92c952", - "thumbnailUrl": "https://via.placeholder.com/150/92c952" - }, - { - "id": 2, - "title": "reprehenderit est deserunt velit ipsam", - "url": "https://via.placeholder.com/600/771796", - "thumbnailUrl": "https://via.placeholder.com/150/771796" - } -] -``` - -Create `AlbumControllerTestcontainersTests.java`: - -```java -package com.testcontainers.demo; - -import static io.restassured.RestAssured.given; -import static org.hamcrest.CoreMatchers.is; -import static org.hamcrest.Matchers.hasSize; -import static org.hamcrest.Matchers.nullValue; - -import io.micronaut.context.ApplicationContext; -import io.micronaut.core.annotation.NonNull; -import io.micronaut.runtime.server.EmbeddedServer; -import io.restassured.RestAssured; -import io.restassured.http.ContentType; -import java.util.Collections; -import java.util.Map; -import org.junit.jupiter.api.Test; -import org.testcontainers.junit.jupiter.Container; -import org.testcontainers.junit.jupiter.Testcontainers; -import org.wiremock.integrations.testcontainers.WireMockContainer; - -@Testcontainers(disabledWithoutDocker = true) -class AlbumControllerTestcontainersTests { - - @Container - static WireMockContainer wiremockServer = new WireMockContainer("wiremock/wiremock:2.35.0") - .withMappingFromResource("mocks-config.json") - .withFileFromResource("album-photos-response.json"); - - @NonNull public Map getProperties() { - return Collections.singletonMap("micronaut.http.services.photosapi.url", wiremockServer.getBaseUrl()); - } - - @Test - void shouldGetAlbumById() { - Long albumId = 1L; - try (EmbeddedServer server = ApplicationContext.run(EmbeddedServer.class, getProperties())) { - RestAssured.port = server.getPort(); - - given().contentType(ContentType.JSON) - .when() - .get("/api/albums/{albumId}", albumId) - .then() - .statusCode(200) - .body("albumId", is(albumId.intValue())) - .body("photos", hasSize(2)); - } - } - - @Test - void shouldReturnServerErrorWhenPhotoServiceCallFailed() { - Long albumId = 2L; - try (EmbeddedServer server = ApplicationContext.run(EmbeddedServer.class, getProperties())) { - RestAssured.port = server.getPort(); - given().contentType(ContentType.JSON) - .when() - .get("/api/albums/{albumId}", albumId) - .then() - .statusCode(500); - } - } - - @Test - void shouldReturnEmptyPhotos() { - Long albumId = 3L; - try (EmbeddedServer server = ApplicationContext.run(EmbeddedServer.class, getProperties())) { - RestAssured.port = server.getPort(); - given().contentType(ContentType.JSON) - .when() - .get("/api/albums/{albumId}", albumId) - .then() - .statusCode(200) - .body("albumId", is(albumId.intValue())) - .body("photos", nullValue()); - } - } -} -``` - -Here's what this test does: - -- `@Testcontainers` and `@Container` annotations start a `WireMockContainer` - using the `wiremock/wiremock:2.35.0` Docker image. -- `withMappingFromResource("mocks-config.json")` loads stub mappings from the - classpath resource. -- `withFileFromResource("album-photos-response.json")` makes the response body - file available to WireMock. -- `getProperties()` overrides the photo service URL to point at the WireMock - container's base URL. -- `shouldGetAlbumById()` verifies that the application returns the expected - album with two photos. -- `shouldReturnServerErrorWhenPhotoServiceCallFailed()` verifies that a 500 - from the photo service propagates to the caller. -- `shouldReturnEmptyPhotos()` verifies the application handles an empty photo - list. diff --git a/content/guides/testcontainers-java-mockserver/_index.md b/content/guides/testcontainers-java-mockserver/_index.md index 0bb73d241e53..fe8a48c137a7 100644 --- a/content/guides/testcontainers-java-mockserver/_index.md +++ b/content/guides/testcontainers-java-mockserver/_index.md @@ -7,14 +7,16 @@ summary: | Learn how to create a Spring Boot application that integrates with external REST APIs, then test those integrations using Testcontainers and MockServer. -toc_min: 1 -toc_max: 2 -tags: [testing-with-docker] -languages: [java] +aliases: + - /guides/testcontainers-java-mockserver/create-project/ + - /guides/testcontainers-java-mockserver/run-tests/ + - /guides/testcontainers-java-mockserver/write-tests/ params: + tags: [testing] time: 20 minutes --- + In this guide, you will learn how to: @@ -32,3 +34,412 @@ In this guide, you will learn how to: > If you're new to Testcontainers, visit the > [Testcontainers overview](https://testcontainers.com/getting-started/) to learn more about > Testcontainers and the benefits of using it. + +## Create the Spring Boot project + +### Set up the project + +Create a Spring Boot project from [Spring Initializr](https://start.spring.io) +by selecting the **Spring Web**, **Spring Reactive Web**, and **Testcontainers** +starters. + +Alternatively, clone the +[guide repository](https://github.com/testcontainers/tc-guide-testing-rest-api-integrations-using-mockserver). + +After generating the project, add the **REST Assured** and **MockServer** +libraries as test dependencies. The key dependencies in `pom.xml` are: + +```xml + + 17 + 2.0.4 + + + + org.springframework.boot + spring-boot-starter-web + + + org.springframework.boot + spring-boot-starter-webflux + + + org.springframework.boot + spring-boot-starter-test + test + + + org.testcontainers + testcontainers-junit-jupiter + test + + + org.testcontainers + testcontainers-mockserver + test + + + org.mock-server + mockserver-netty + 5.15.0 + test + + + io.rest-assured + rest-assured + test + + +``` + +Using the Testcontainers BOM (Bill of Materials) is recommended so that you +don't have to repeat the version for every Testcontainers module dependency. + +This guide builds an application that manages video albums. A third-party REST +API handles photo assets. For demonstration purposes, the application uses the +publicly available [JSONPlaceholder](https://jsonplaceholder.typicode.com/) API +as a photo service. + +The application exposes a `GET /api/albums/{albumId}` endpoint that calls the +photo service to fetch photos for a given album. +[MockServer](https://www.mock-server.com/) is a library for mocking HTTP-based +services. Testcontainers provides a +[MockServer module](https://java.testcontainers.org/modules/mockserver/) that +runs MockServer as a Docker container. + +### Create the Album and Photo models + +Create `Album.java` using Java records: + +```java +package com.testcontainers.demo; + +import java.util.List; + +public record Album(Long albumId, List photos) {} + +record Photo(Long id, String title, String url, String thumbnailUrl) {} +``` + +### Create the PhotoServiceClient interface + +Spring Framework 6 introduced +[declarative HTTP client support](https://docs.spring.io/spring-framework/reference/integration/rest-clients.html#rest-http-interface). +Create an interface with a method that fetches photos for a given album ID: + +```java +package com.testcontainers.demo; + +import java.util.List; +import org.springframework.web.bind.annotation.PathVariable; +import org.springframework.web.service.annotation.GetExchange; + +interface PhotoServiceClient { + @GetExchange("/albums/{albumId}/photos") + List getPhotos(@PathVariable Long albumId); +} +``` + +### Register PhotoServiceClient as a bean + +To generate a runtime implementation of `PhotoServiceClient`, register it as a +Spring bean using `HttpServiceProxyFactory`. The factory requires an +`HttpClientAdapter` implementation. Spring Boot provides `WebClientAdapter` as +part of the `spring-webflux` library: + +```java +package com.testcontainers.demo; + +import org.springframework.beans.factory.annotation.Value; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.web.reactive.function.client.WebClient; +import org.springframework.web.reactive.function.client.support.WebClientAdapter; +import org.springframework.web.service.invoker.HttpServiceProxyFactory; + +@Configuration +public class AppConfig { + + @Bean + public PhotoServiceClient photoServiceClient( + @Value("${photos.api.base-url}") String photosApiBaseUrl + ) { + WebClient client = WebClient.builder().baseUrl(photosApiBaseUrl).build(); + HttpServiceProxyFactory factory = HttpServiceProxyFactory + .builder(WebClientAdapter.forClient(client)) + .build(); + return factory.createClient(PhotoServiceClient.class); + } +} +``` + +The photo service base URL is externalized as a configuration property. Add the +following entry to `src/main/resources/application.properties`: + +```properties +photos.api.base-url=https://jsonplaceholder.typicode.com +``` + +### Create the REST API endpoint + +Create `AlbumController.java`: + +```java +package com.testcontainers.demo; + +import java.util.List; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.http.ResponseEntity; +import org.springframework.web.bind.annotation.GetMapping; +import org.springframework.web.bind.annotation.PathVariable; +import org.springframework.web.bind.annotation.RequestMapping; +import org.springframework.web.bind.annotation.RestController; +import org.springframework.web.reactive.function.client.WebClientResponseException; + +@RestController +@RequestMapping("/api") +class AlbumController { + + private static final Logger logger = LoggerFactory.getLogger( + AlbumController.class + ); + + private final PhotoServiceClient photoServiceClient; + + AlbumController(PhotoServiceClient photoServiceClient) { + this.photoServiceClient = photoServiceClient; + } + + @GetMapping("/albums/{albumId}") + public ResponseEntity getAlbumById(@PathVariable Long albumId) { + try { + List photos = photoServiceClient.getPhotos(albumId); + return ResponseEntity.ok(new Album(albumId, photos)); + } catch (WebClientResponseException e) { + logger.error("Failed to get photos", e); + return new ResponseEntity<>(e.getStatusCode()); + } + } +} +``` + +This endpoint calls the photo service for a given album ID and returns a +response like: + +```json +{ + "albumId": 1, + "photos": [ + { + "id": 51, + "title": "non sunt voluptatem placeat consequuntur rem incidunt", + "url": "https://via.placeholder.com/600/8e973b", + "thumbnailUrl": "https://via.placeholder.com/150/8e973b" + }, + { + "id": 52, + "title": "eveniet pariatur quia nobis reiciendis laboriosam ea", + "url": "https://via.placeholder.com/600/121fa4", + "thumbnailUrl": "https://via.placeholder.com/150/121fa4" + } + ] +} +``` + +## Write tests with Testcontainers MockServer + +Mocking external API interactions at the HTTP protocol level, rather than +mocking Java methods, lets you verify marshalling and unmarshalling behavior and +simulate network issues. + +Testcontainers provides a MockServer module that starts a +[MockServer](https://www.mock-server.com/) instance inside a Docker container. +You can then use `MockServerClient` to configure mock expectations. + +### Write the test + +Create `AlbumControllerTest.java`: + +```java +package com.testcontainers.demo; + +import static io.restassured.RestAssured.given; +import static org.hamcrest.CoreMatchers.is; +import static org.hamcrest.Matchers.hasSize; +import static org.mockserver.model.HttpRequest.request; +import static org.mockserver.model.HttpResponse.response; +import static org.mockserver.model.JsonBody.json; + +import io.restassured.RestAssured; +import io.restassured.http.ContentType; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.mockserver.client.MockServerClient; +import org.mockserver.model.Header; +import org.mockserver.verify.VerificationTimes; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.web.server.LocalServerPort; +import org.springframework.test.context.DynamicPropertyRegistry; +import org.springframework.test.context.DynamicPropertySource; +import org.testcontainers.mockserver.MockServerContainer; +import org.testcontainers.junit.jupiter.Container; +import org.testcontainers.junit.jupiter.Testcontainers; + +@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT) +@Testcontainers +class AlbumControllerTest { + + @LocalServerPort + private Integer port; + + @Container + static MockServerContainer mockServerContainer = + new MockServerContainer("mockserver/mockserver:5.15.0"); + + static MockServerClient mockServerClient; + + @DynamicPropertySource + static void overrideProperties(DynamicPropertyRegistry registry) { + mockServerClient = + new MockServerClient( + mockServerContainer.getHost(), + mockServerContainer.getServerPort() + ); + registry.add("photos.api.base-url", mockServerContainer::getEndpoint); + } + + @BeforeEach + void setUp() { + RestAssured.port = port; + mockServerClient.reset(); + } + + @Test + void shouldGetAlbumById() { + Long albumId = 1L; + + mockServerClient + .when( + request().withMethod("GET").withPath("/albums/" + albumId + "/photos") + ) + .respond( + response() + .withStatusCode(200) + .withHeaders( + new Header("Content-Type", "application/json; charset=utf-8") + ) + .withBody( + json( + """ + [ + { + "id": 1, + "title": "accusamus beatae ad facilis cum similique qui sunt", + "url": "https://via.placeholder.com/600/92c952", + "thumbnailUrl": "https://via.placeholder.com/150/92c952" + }, + { + "id": 2, + "title": "reprehenderit est deserunt velit ipsam", + "url": "https://via.placeholder.com/600/771796", + "thumbnailUrl": "https://via.placeholder.com/150/771796" + } + ] + """ + ) + ) + ); + + given() + .contentType(ContentType.JSON) + .when() + .get("/api/albums/{albumId}", albumId) + .then() + .statusCode(200) + .body("albumId", is(albumId.intValue())) + .body("photos", hasSize(2)); + + verifyMockServerRequest("GET", "/albums/" + albumId + "/photos", 1); + } + + @Test + void shouldReturn404StatusWhenAlbumNotFound() { + Long albumId = 1L; + mockServerClient + .when( + request().withMethod("GET").withPath("/albums/" + albumId + "/photos") + ) + .respond(response().withStatusCode(404)); + + given() + .contentType(ContentType.JSON) + .when() + .get("/api/albums/{albumId}", albumId) + .then() + .statusCode(404); + + verifyMockServerRequest("GET", "/albums/" + albumId + "/photos", 1); + } + + private void verifyMockServerRequest(String method, String path, int times) { + mockServerClient.verify( + request().withMethod(method).withPath(path), + VerificationTimes.exactly(times) + ); + } +} +``` + +Here's what the test does: + +- `@SpringBootTest` starts the full application on a random port. +- The `@Testcontainers` and `@Container` annotations start a + `MockServerContainer` and create a `MockServerClient` connected to it. +- `@DynamicPropertySource` overrides `photos.api.base-url` to point at the + MockServer endpoint, so the application talks to MockServer instead of the + real photo service. +- `@BeforeEach` resets the `MockServerClient` before every test so that + expectations from one test don't affect another. +- `shouldGetAlbumById()` configures a mock response for + `/albums/{albumId}/photos`, sends a request to the application's + `/api/albums/{albumId}` endpoint, and verifies the response body. It also + uses `mockServerClient.verify()` to confirm that the expected API call + reached MockServer. +- `shouldReturn404StatusWhenAlbumNotFound()` configures MockServer to return a + 404 status and verifies the application propagates that status to the caller. + +## Run tests and next steps + +### Run the tests + +```console +$ ./mvnw test +``` + +Or with Gradle: + +```console +$ ./gradlew test +``` + +You should see the MockServer Docker container start in the console output. It +acts as the photo service, serving mock responses based on the configured +expectations. All tests should pass. + +### Summary + +You built a Spring Boot application that integrates with an external REST API +using declarative HTTP clients, then tested that integration using the +Testcontainers MockServer module. Testing at the HTTP protocol level instead of +mocking Java methods lets you catch serialization issues and simulate realistic +failure scenarios. + +To learn more about Testcontainers, visit the +[Testcontainers overview](https://testcontainers.com/getting-started/). + +### Further reading + +- [Testcontainers MockServer module](https://java.testcontainers.org/modules/mockserver/) +- [MockServer documentation](https://www.mock-server.com/) +- [Testcontainers JUnit 5 quickstart](https://java.testcontainers.org/quickstart/junit_5_quickstart/) diff --git a/content/guides/testcontainers-java-mockserver/create-project.md b/content/guides/testcontainers-java-mockserver/create-project.md deleted file mode 100644 index 63edcf3d8866..000000000000 --- a/content/guides/testcontainers-java-mockserver/create-project.md +++ /dev/null @@ -1,217 +0,0 @@ ---- -title: Create the Spring Boot project -linkTitle: Create the project -description: Set up a Spring Boot project with an external REST API integration using declarative HTTP clients. -keywords: testcontainers, java, spring boot, mockserver, rest api, project setup -weight: 10 ---- - -## Set up the project - -Create a Spring Boot project from [Spring Initializr](https://start.spring.io) -by selecting the **Spring Web**, **Spring Reactive Web**, and **Testcontainers** -starters. - -Alternatively, clone the -[guide repository](https://github.com/testcontainers/tc-guide-testing-rest-api-integrations-using-mockserver). - -After generating the project, add the **REST Assured** and **MockServer** -libraries as test dependencies. The key dependencies in `pom.xml` are: - -```xml - - 17 - 2.0.4 - - - - org.springframework.boot - spring-boot-starter-web - - - org.springframework.boot - spring-boot-starter-webflux - - - org.springframework.boot - spring-boot-starter-test - test - - - org.testcontainers - testcontainers-junit-jupiter - test - - - org.testcontainers - testcontainers-mockserver - test - - - org.mock-server - mockserver-netty - 5.15.0 - test - - - io.rest-assured - rest-assured - test - - -``` - -Using the Testcontainers BOM (Bill of Materials) is recommended so that you -don't have to repeat the version for every Testcontainers module dependency. - -This guide builds an application that manages video albums. A third-party REST -API handles photo assets. For demonstration purposes, the application uses the -publicly available [JSONPlaceholder](https://jsonplaceholder.typicode.com/) API -as a photo service. - -The application exposes a `GET /api/albums/{albumId}` endpoint that calls the -photo service to fetch photos for a given album. -[MockServer](https://www.mock-server.com/) is a library for mocking HTTP-based -services. Testcontainers provides a -[MockServer module](https://java.testcontainers.org/modules/mockserver/) that -runs MockServer as a Docker container. - -## Create the Album and Photo models - -Create `Album.java` using Java records: - -```java -package com.testcontainers.demo; - -import java.util.List; - -public record Album(Long albumId, List photos) {} - -record Photo(Long id, String title, String url, String thumbnailUrl) {} -``` - -## Create the PhotoServiceClient interface - -Spring Framework 6 introduced -[declarative HTTP client support](https://docs.spring.io/spring-framework/reference/integration/rest-clients.html#rest-http-interface). -Create an interface with a method that fetches photos for a given album ID: - -```java -package com.testcontainers.demo; - -import java.util.List; -import org.springframework.web.bind.annotation.PathVariable; -import org.springframework.web.service.annotation.GetExchange; - -interface PhotoServiceClient { - @GetExchange("/albums/{albumId}/photos") - List getPhotos(@PathVariable Long albumId); -} -``` - -## Register PhotoServiceClient as a bean - -To generate a runtime implementation of `PhotoServiceClient`, register it as a -Spring bean using `HttpServiceProxyFactory`. The factory requires an -`HttpClientAdapter` implementation. Spring Boot provides `WebClientAdapter` as -part of the `spring-webflux` library: - -```java -package com.testcontainers.demo; - -import org.springframework.beans.factory.annotation.Value; -import org.springframework.context.annotation.Bean; -import org.springframework.context.annotation.Configuration; -import org.springframework.web.reactive.function.client.WebClient; -import org.springframework.web.reactive.function.client.support.WebClientAdapter; -import org.springframework.web.service.invoker.HttpServiceProxyFactory; - -@Configuration -public class AppConfig { - - @Bean - public PhotoServiceClient photoServiceClient( - @Value("${photos.api.base-url}") String photosApiBaseUrl - ) { - WebClient client = WebClient.builder().baseUrl(photosApiBaseUrl).build(); - HttpServiceProxyFactory factory = HttpServiceProxyFactory - .builder(WebClientAdapter.forClient(client)) - .build(); - return factory.createClient(PhotoServiceClient.class); - } -} -``` - -The photo service base URL is externalized as a configuration property. Add the -following entry to `src/main/resources/application.properties`: - -```properties -photos.api.base-url=https://jsonplaceholder.typicode.com -``` - -## Create the REST API endpoint - -Create `AlbumController.java`: - -```java -package com.testcontainers.demo; - -import java.util.List; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.springframework.http.ResponseEntity; -import org.springframework.web.bind.annotation.GetMapping; -import org.springframework.web.bind.annotation.PathVariable; -import org.springframework.web.bind.annotation.RequestMapping; -import org.springframework.web.bind.annotation.RestController; -import org.springframework.web.reactive.function.client.WebClientResponseException; - -@RestController -@RequestMapping("/api") -class AlbumController { - - private static final Logger logger = LoggerFactory.getLogger( - AlbumController.class - ); - - private final PhotoServiceClient photoServiceClient; - - AlbumController(PhotoServiceClient photoServiceClient) { - this.photoServiceClient = photoServiceClient; - } - - @GetMapping("/albums/{albumId}") - public ResponseEntity getAlbumById(@PathVariable Long albumId) { - try { - List photos = photoServiceClient.getPhotos(albumId); - return ResponseEntity.ok(new Album(albumId, photos)); - } catch (WebClientResponseException e) { - logger.error("Failed to get photos", e); - return new ResponseEntity<>(e.getStatusCode()); - } - } -} -``` - -This endpoint calls the photo service for a given album ID and returns a -response like: - -```json -{ - "albumId": 1, - "photos": [ - { - "id": 51, - "title": "non sunt voluptatem placeat consequuntur rem incidunt", - "url": "https://via.placeholder.com/600/8e973b", - "thumbnailUrl": "https://via.placeholder.com/150/8e973b" - }, - { - "id": 52, - "title": "eveniet pariatur quia nobis reiciendis laboriosam ea", - "url": "https://via.placeholder.com/600/121fa4", - "thumbnailUrl": "https://via.placeholder.com/150/121fa4" - } - ] -} -``` diff --git a/content/guides/testcontainers-java-mockserver/run-tests.md b/content/guides/testcontainers-java-mockserver/run-tests.md deleted file mode 100644 index ab8e071d5c29..000000000000 --- a/content/guides/testcontainers-java-mockserver/run-tests.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -title: Run tests and next steps -linkTitle: Run tests -description: Run your Testcontainers MockServer integration tests and explore next steps. -keywords: testcontainers, java, spring boot, mockserver, integration testing -weight: 30 ---- - -## Run the tests - -```console -$ ./mvnw test -``` - -Or with Gradle: - -```console -$ ./gradlew test -``` - -You should see the MockServer Docker container start in the console output. It -acts as the photo service, serving mock responses based on the configured -expectations. All tests should pass. - -## Summary - -You built a Spring Boot application that integrates with an external REST API -using declarative HTTP clients, then tested that integration using the -Testcontainers MockServer module. Testing at the HTTP protocol level instead of -mocking Java methods lets you catch serialization issues and simulate realistic -failure scenarios. - -To learn more about Testcontainers, visit the -[Testcontainers overview](https://testcontainers.com/getting-started/). - -## Further reading - -- [Testcontainers MockServer module](https://java.testcontainers.org/modules/mockserver/) -- [MockServer documentation](https://www.mock-server.com/) -- [Testcontainers JUnit 5 quickstart](https://java.testcontainers.org/quickstart/junit_5_quickstart/) diff --git a/content/guides/testcontainers-java-mockserver/write-tests.md b/content/guides/testcontainers-java-mockserver/write-tests.md deleted file mode 100644 index e48654bd8208..000000000000 --- a/content/guides/testcontainers-java-mockserver/write-tests.md +++ /dev/null @@ -1,167 +0,0 @@ ---- -title: Write tests with Testcontainers MockServer -linkTitle: Write tests -description: Test external REST API integrations using the Testcontainers MockServer module and REST Assured. -keywords: testcontainers, java, spring boot, mockserver, rest assured, integration testing -weight: 20 ---- - -Mocking external API interactions at the HTTP protocol level, rather than -mocking Java methods, lets you verify marshalling and unmarshalling behavior and -simulate network issues. - -Testcontainers provides a MockServer module that starts a -[MockServer](https://www.mock-server.com/) instance inside a Docker container. -You can then use `MockServerClient` to configure mock expectations. - -## Write the test - -Create `AlbumControllerTest.java`: - -```java -package com.testcontainers.demo; - -import static io.restassured.RestAssured.given; -import static org.hamcrest.CoreMatchers.is; -import static org.hamcrest.Matchers.hasSize; -import static org.mockserver.model.HttpRequest.request; -import static org.mockserver.model.HttpResponse.response; -import static org.mockserver.model.JsonBody.json; - -import io.restassured.RestAssured; -import io.restassured.http.ContentType; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.Test; -import org.mockserver.client.MockServerClient; -import org.mockserver.model.Header; -import org.mockserver.verify.VerificationTimes; -import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.test.web.server.LocalServerPort; -import org.springframework.test.context.DynamicPropertyRegistry; -import org.springframework.test.context.DynamicPropertySource; -import org.testcontainers.mockserver.MockServerContainer; -import org.testcontainers.junit.jupiter.Container; -import org.testcontainers.junit.jupiter.Testcontainers; - -@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT) -@Testcontainers -class AlbumControllerTest { - - @LocalServerPort - private Integer port; - - @Container - static MockServerContainer mockServerContainer = - new MockServerContainer("mockserver/mockserver:5.15.0"); - - static MockServerClient mockServerClient; - - @DynamicPropertySource - static void overrideProperties(DynamicPropertyRegistry registry) { - mockServerClient = - new MockServerClient( - mockServerContainer.getHost(), - mockServerContainer.getServerPort() - ); - registry.add("photos.api.base-url", mockServerContainer::getEndpoint); - } - - @BeforeEach - void setUp() { - RestAssured.port = port; - mockServerClient.reset(); - } - - @Test - void shouldGetAlbumById() { - Long albumId = 1L; - - mockServerClient - .when( - request().withMethod("GET").withPath("/albums/" + albumId + "/photos") - ) - .respond( - response() - .withStatusCode(200) - .withHeaders( - new Header("Content-Type", "application/json; charset=utf-8") - ) - .withBody( - json( - """ - [ - { - "id": 1, - "title": "accusamus beatae ad facilis cum similique qui sunt", - "url": "https://via.placeholder.com/600/92c952", - "thumbnailUrl": "https://via.placeholder.com/150/92c952" - }, - { - "id": 2, - "title": "reprehenderit est deserunt velit ipsam", - "url": "https://via.placeholder.com/600/771796", - "thumbnailUrl": "https://via.placeholder.com/150/771796" - } - ] - """ - ) - ) - ); - - given() - .contentType(ContentType.JSON) - .when() - .get("/api/albums/{albumId}", albumId) - .then() - .statusCode(200) - .body("albumId", is(albumId.intValue())) - .body("photos", hasSize(2)); - - verifyMockServerRequest("GET", "/albums/" + albumId + "/photos", 1); - } - - @Test - void shouldReturn404StatusWhenAlbumNotFound() { - Long albumId = 1L; - mockServerClient - .when( - request().withMethod("GET").withPath("/albums/" + albumId + "/photos") - ) - .respond(response().withStatusCode(404)); - - given() - .contentType(ContentType.JSON) - .when() - .get("/api/albums/{albumId}", albumId) - .then() - .statusCode(404); - - verifyMockServerRequest("GET", "/albums/" + albumId + "/photos", 1); - } - - private void verifyMockServerRequest(String method, String path, int times) { - mockServerClient.verify( - request().withMethod(method).withPath(path), - VerificationTimes.exactly(times) - ); - } -} -``` - -Here's what the test does: - -- `@SpringBootTest` starts the full application on a random port. -- The `@Testcontainers` and `@Container` annotations start a - `MockServerContainer` and create a `MockServerClient` connected to it. -- `@DynamicPropertySource` overrides `photos.api.base-url` to point at the - MockServer endpoint, so the application talks to MockServer instead of the - real photo service. -- `@BeforeEach` resets the `MockServerClient` before every test so that - expectations from one test don't affect another. -- `shouldGetAlbumById()` configures a mock response for - `/albums/{albumId}/photos`, sends a request to the application's - `/api/albums/{albumId}` endpoint, and verifies the response body. It also - uses `mockServerClient.verify()` to confirm that the expected API call - reached MockServer. -- `shouldReturn404StatusWhenAlbumNotFound()` configures MockServer to return a - 404 status and verifies the application propagates that status to the caller. diff --git a/content/guides/testcontainers-java-quarkus/_index.md b/content/guides/testcontainers-java-quarkus/_index.md index 2645bb66f6b4..71b1447de4d3 100644 --- a/content/guides/testcontainers-java-quarkus/_index.md +++ b/content/guides/testcontainers-java-quarkus/_index.md @@ -6,14 +6,16 @@ keywords: testcontainers, java, quarkus, testing, postgresql, rest api, rest ass summary: | Learn how to create a Quarkus REST API with Hibernate ORM with Panache and PostgreSQL, then test it using Quarkus Dev Services, Testcontainers, and REST Assured. -toc_min: 1 -toc_max: 2 -tags: [testing-with-docker] -languages: [java] +aliases: + - /guides/testcontainers-java-quarkus/create-project/ + - /guides/testcontainers-java-quarkus/run-tests/ + - /guides/testcontainers-java-quarkus/write-tests/ params: + tags: [testing] time: 25 minutes --- + In this guide, you'll learn how to: @@ -35,3 +37,438 @@ In this guide, you'll learn how to: > If you're new to Testcontainers, visit the > [Testcontainers overview](https://testcontainers.com/getting-started/) to learn more about > Testcontainers and the benefits of using it. + +## Create the Quarkus project + +### Set up the project + +Create a Quarkus project from [code.quarkus.io](https://code.quarkus.io/) by +selecting the **RESTEasy Classic**, **RESTEasy Classic Jackson**, +**Hibernate Validator**, **Hibernate ORM with Panache**, **JDBC Driver - +PostgreSQL**, and **Flyway** extensions. + +Alternatively, clone the +[guide repository](https://github.com/testcontainers/tc-guide-testcontainers-in-quarkus-applications). + +The key dependencies in `pom.xml` are: + +```xml + + 3.22.3 + + + + io.quarkus + quarkus-hibernate-orm-panache + + + io.quarkus + quarkus-flyway + + + io.quarkus + quarkus-hibernate-validator + + + io.quarkus + quarkus-resteasy + + + io.quarkus + quarkus-resteasy-jackson + + + io.quarkus + quarkus-jdbc-postgresql + + + io.quarkus + quarkus-junit5 + test + + + io.rest-assured + rest-assured + test + + +``` + +### Create the JPA entity + +Hibernate ORM with Panache supports the Active Record pattern and the +Repository pattern to simplify JPA usage. This guide uses the Active Record +pattern. + +Create `Customer.java` by extending `PanacheEntity`. This gives the entity +built-in persistence methods such as `persist()`, `listAll()`, and +`findById()`. + +```java +package com.testcontainers.demo; + +import io.quarkus.hibernate.orm.panache.PanacheEntity; +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.Table; + +@Entity +@Table(name = "customers") +public class Customer extends PanacheEntity { + + @Column(nullable = false) + public String name; + + @Column(nullable = false, unique = true) + public String email; + + public Customer() {} + + public Customer(Long id, String name, String email) { + this.id = id; + this.name = name; + this.email = email; + } +} +``` + +### Create the CustomerService CDI bean + +Create a `CustomerService` class annotated with `@ApplicationScoped` and +`@Transactional` to handle persistence operations: + +```java +package com.testcontainers.demo; + +import jakarta.enterprise.context.ApplicationScoped; +import jakarta.transaction.Transactional; +import java.util.List; + +@ApplicationScoped +@Transactional +public class CustomerService { + + public List getAll() { + return Customer.listAll(); + } + + public Customer create(Customer customer) { + customer.persist(); + return customer; + } +} +``` + +### Add the Flyway database migration script + +Create `src/main/resources/db/migration/V1__init_database.sql`: + +```sql +create sequence customers_seq start with 1 increment by 50; + +create table customers +( + id bigint DEFAULT nextval('customers_seq') not null, + name varchar not null, + email varchar not null, + primary key (id) +); + +insert into customers(name, email) +values ('john', 'john@mail.com'), + ('rambo', 'rambo@mail.com'); +``` + +Enable Flyway migrations in `src/main/resources/application.properties`: + +```properties +quarkus.flyway.migrate-at-start=true +``` + +### Create the REST API endpoints + +Create `CustomerResource.java` with endpoints for fetching all customers and +creating a customer: + +```java +package com.testcontainers.demo; + +import jakarta.ws.rs.Consumes; +import jakarta.ws.rs.GET; +import jakarta.ws.rs.POST; +import jakarta.ws.rs.Path; +import jakarta.ws.rs.Produces; +import jakarta.ws.rs.core.MediaType; +import jakarta.ws.rs.core.Response; +import java.util.List; + +@Path("/api/customers") +@Produces(MediaType.APPLICATION_JSON) +@Consumes(MediaType.APPLICATION_JSON) +public class CustomerResource { + private final CustomerService customerService; + + public CustomerResource(CustomerService customerService) { + this.customerService = customerService; + } + + @GET + public List getAllCustomers() { + return customerService.getAll(); + } + + @POST + public Response createCustomer(Customer customer) { + var savedCustomer = customerService.create(customer); + return Response.status(Response.Status.CREATED).entity(savedCustomer).build(); + } +} +``` + +## Write tests with Testcontainers + +### Quarkus Dev Services + +Quarkus Dev Services automatically provisions unconfigured services in +development and test mode. When you include an extension and don't configure it, +Quarkus starts the relevant service using +[Testcontainers](https://www.testcontainers.org/) behind the scenes and wires +the application to use that service. + +> [!NOTE] +> Dev Services requires a +> [supported Docker environment](https://www.testcontainers.org/supported_docker_environment/). + +Quarkus Dev Services supports most commonly used services like SQL databases, +Kafka, RabbitMQ, Redis, and MongoDB. For more information, see the +[Quarkus Dev Services guide](https://quarkus.io/guides/dev-services). + +### Write tests for the API endpoints + +Test the `GET /api/customers` and `POST /api/customers` endpoints using REST +Assured. The `io.rest-assured:rest-assured` library was already added as a test +dependency when you generated the project. + +Create `CustomerResourceTest.java` and annotate it with `@QuarkusTest`. This +bootstraps the application along with the required services using Dev Services. +Because you haven't configured datasource properties, Dev Services automatically +starts a PostgreSQL database using Testcontainers. + +```java +package com.testcontainers.demo; + +import static io.restassured.RestAssured.given; +import static org.hamcrest.CoreMatchers.is; +import static org.junit.jupiter.api.Assertions.assertFalse; + +import io.quarkus.test.junit.QuarkusTest; +import io.restassured.common.mapper.TypeRef; +import io.restassured.http.ContentType; +import java.util.List; +import org.junit.jupiter.api.Test; + +@QuarkusTest +class CustomerResourceTest { + + @Test + void shouldGetAllCustomers() { + List customers = given().when() + .get("/api/customers") + .then() + .statusCode(200) + .extract() + .as(new TypeRef<>() {}); + assertFalse(customers.isEmpty()); + } + + @Test + void shouldCreateCustomerSuccessfully() { + Customer customer = new Customer(null, "John", "john@gmail.com"); + given().contentType(ContentType.JSON) + .body(customer) + .when() + .post("/api/customers") + .then() + .statusCode(201) + .body("name", is("John")) + .body("email", is("john@gmail.com")); + } +} +``` + +Here's what the test does: + +- `@QuarkusTest` starts the full Quarkus application with Dev Services enabled. +- Dev Services starts a PostgreSQL container using Testcontainers and configures + the datasource automatically. +- `shouldGetAllCustomers()` calls `GET /api/customers` and verifies that seeded + data from the Flyway migration is returned. +- `shouldCreateCustomerSuccessfully()` sends a `POST /api/customers` request and + verifies the response contains the created customer data. + +### Customize test configuration + +By default, the Quarkus test instance starts on port 8081 and uses a +`postgres:14` Docker image. Customize both by adding these properties to +`src/main/resources/application.properties`: + +```properties +quarkus.http.test-port=0 +quarkus.datasource.devservices.image-name=postgres:15.2-alpine +``` + +Setting `quarkus.http.test-port=0` starts the application on a random available +port, avoiding port conflicts. The `devservices.image-name` property lets you +pin the PostgreSQL image to a specific version that matches production. + +### Test with services not supported by Dev Services + +Your application might use a service that Dev Services doesn't support out of +the box. In that case, use `QuarkusTestResourceLifecycleManager` to start the +service before the Quarkus application starts for testing. + +For example, suppose the application uses CockroachDB. First, add the +CockroachDB Testcontainers module dependency: + +```xml + + org.testcontainers + cockroachdb + test + +``` + +Create a `CockroachDBTestResource` that implements +`QuarkusTestResourceLifecycleManager`: + +```java +package com.testcontainers.demo; + +import io.quarkus.test.common.QuarkusTestResourceLifecycleManager; +import java.util.HashMap; +import java.util.Map; +import org.testcontainers.containers.CockroachContainer; + +public class CockroachDBTestResource implements QuarkusTestResourceLifecycleManager { + + CockroachContainer cockroachdb; + + @Override + public Map start() { + cockroachdb = new CockroachContainer("cockroachdb/cockroach:v22.2.0"); + cockroachdb.start(); + Map conf = new HashMap<>(); + conf.put("quarkus.datasource.jdbc.url", cockroachdb.getJdbcUrl()); + conf.put("quarkus.datasource.username", cockroachdb.getUsername()); + conf.put("quarkus.datasource.password", cockroachdb.getPassword()); + return conf; + } + + @Override + public void stop() { + cockroachdb.stop(); + } +} +``` + +Use the `CockroachDBTestResource` with `@QuarkusTestResource` in a test class: + +```java +package com.testcontainers.demo; + +import static io.restassured.RestAssured.given; +import static org.junit.jupiter.api.Assertions.assertFalse; + +import io.quarkus.test.common.QuarkusTestResource; +import io.quarkus.test.junit.QuarkusTest; +import io.restassured.common.mapper.TypeRef; +import java.util.List; +import org.junit.jupiter.api.Test; + +@QuarkusTest +@QuarkusTestResource(value = CockroachDBTestResource.class, restrictToAnnotatedClass = true) +class CockroachDBTest { + + @Test + void shouldGetAllCustomers() { + List customers = given().when() + .get("/api/customers") + .then() + .statusCode(200) + .extract() + .as(new TypeRef<>() {}); + assertFalse(customers.isEmpty()); + } +} +``` + +The `restrictToAnnotatedClass = true` attribute ensures the CockroachDB +container only starts when running this specific test class, rather than being +activated for all tests. + +## Run tests and next steps + +### Run the tests + +```console +$ ./mvnw test +``` + +Or with Gradle: + +```console +$ ./gradlew test +``` + +You should see the PostgreSQL Docker container start and all tests pass. After +the tests finish, the container stops and is removed automatically. + +### Run the application locally + +Quarkus Dev Services automatically provisions unconfigured services in +development mode. Start the Quarkus application in dev mode: + +```console +$ ./mvnw compile quarkus:dev +``` + +Or with Gradle: + +```console +$ ./gradlew quarkusDev +``` + +Dev Services starts a PostgreSQL container automatically. If you're running a +PostgreSQL database on your system and want to use that instead, configure the +datasource properties in `src/main/resources/application.properties`: + +```properties +quarkus.datasource.jdbc.url=jdbc:postgresql://localhost:5432/postgres +quarkus.datasource.username=postgres +quarkus.datasource.password=postgres +``` + +When these properties are set explicitly, Dev Services doesn't provision the +database container and instead connects to the configured database. + +### Summary + +Quarkus Dev Services improves the developer experience by automatically +provisioning the required services using Testcontainers during development and +testing. This guide covered: + +- Building a REST API using JAX-RS with Hibernate ORM with Panache +- Testing API endpoints using REST Assured with Dev Services handling database + provisioning +- Using `QuarkusTestResourceLifecycleManager` for services not supported by Dev + Services +- Running the application locally with Dev Services + +To learn more about Testcontainers, visit the +[Testcontainers overview](https://testcontainers.com/getting-started/). + +### Further reading + +- [Quarkus Dev Services overview](https://quarkus.io/guides/dev-services) +- [Quarkus testing guide](https://quarkus.io/guides/getting-started-testing) +- [Testcontainers Postgres module](https://java.testcontainers.org/modules/databases/postgres/) diff --git a/content/guides/testcontainers-java-quarkus/create-project.md b/content/guides/testcontainers-java-quarkus/create-project.md deleted file mode 100644 index 52d5158a8162..000000000000 --- a/content/guides/testcontainers-java-quarkus/create-project.md +++ /dev/null @@ -1,192 +0,0 @@ ---- -title: Create the Quarkus project -linkTitle: Create the project -description: Set up a Quarkus project with Hibernate ORM with Panache, PostgreSQL, Flyway, and REST API endpoints. -keywords: testcontainers, java, quarkus, postgresql, hibernate, flyway, project setup -weight: 10 ---- - -## Set up the project - -Create a Quarkus project from [code.quarkus.io](https://code.quarkus.io/) by -selecting the **RESTEasy Classic**, **RESTEasy Classic Jackson**, -**Hibernate Validator**, **Hibernate ORM with Panache**, **JDBC Driver - -PostgreSQL**, and **Flyway** extensions. - -Alternatively, clone the -[guide repository](https://github.com/testcontainers/tc-guide-testcontainers-in-quarkus-applications). - -The key dependencies in `pom.xml` are: - -```xml - - 3.22.3 - - - - io.quarkus - quarkus-hibernate-orm-panache - - - io.quarkus - quarkus-flyway - - - io.quarkus - quarkus-hibernate-validator - - - io.quarkus - quarkus-resteasy - - - io.quarkus - quarkus-resteasy-jackson - - - io.quarkus - quarkus-jdbc-postgresql - - - io.quarkus - quarkus-junit5 - test - - - io.rest-assured - rest-assured - test - - -``` - -## Create the JPA entity - -Hibernate ORM with Panache supports the Active Record pattern and the -Repository pattern to simplify JPA usage. This guide uses the Active Record -pattern. - -Create `Customer.java` by extending `PanacheEntity`. This gives the entity -built-in persistence methods such as `persist()`, `listAll()`, and -`findById()`. - -```java -package com.testcontainers.demo; - -import io.quarkus.hibernate.orm.panache.PanacheEntity; -import jakarta.persistence.Column; -import jakarta.persistence.Entity; -import jakarta.persistence.Table; - -@Entity -@Table(name = "customers") -public class Customer extends PanacheEntity { - - @Column(nullable = false) - public String name; - - @Column(nullable = false, unique = true) - public String email; - - public Customer() {} - - public Customer(Long id, String name, String email) { - this.id = id; - this.name = name; - this.email = email; - } -} -``` - -## Create the CustomerService CDI bean - -Create a `CustomerService` class annotated with `@ApplicationScoped` and -`@Transactional` to handle persistence operations: - -```java -package com.testcontainers.demo; - -import jakarta.enterprise.context.ApplicationScoped; -import jakarta.transaction.Transactional; -import java.util.List; - -@ApplicationScoped -@Transactional -public class CustomerService { - - public List getAll() { - return Customer.listAll(); - } - - public Customer create(Customer customer) { - customer.persist(); - return customer; - } -} -``` - -## Add the Flyway database migration script - -Create `src/main/resources/db/migration/V1__init_database.sql`: - -```sql -create sequence customers_seq start with 1 increment by 50; - -create table customers -( - id bigint DEFAULT nextval('customers_seq') not null, - name varchar not null, - email varchar not null, - primary key (id) -); - -insert into customers(name, email) -values ('john', 'john@mail.com'), - ('rambo', 'rambo@mail.com'); -``` - -Enable Flyway migrations in `src/main/resources/application.properties`: - -```properties -quarkus.flyway.migrate-at-start=true -``` - -## Create the REST API endpoints - -Create `CustomerResource.java` with endpoints for fetching all customers and -creating a customer: - -```java -package com.testcontainers.demo; - -import jakarta.ws.rs.Consumes; -import jakarta.ws.rs.GET; -import jakarta.ws.rs.POST; -import jakarta.ws.rs.Path; -import jakarta.ws.rs.Produces; -import jakarta.ws.rs.core.MediaType; -import jakarta.ws.rs.core.Response; -import java.util.List; - -@Path("/api/customers") -@Produces(MediaType.APPLICATION_JSON) -@Consumes(MediaType.APPLICATION_JSON) -public class CustomerResource { - private final CustomerService customerService; - - public CustomerResource(CustomerService customerService) { - this.customerService = customerService; - } - - @GET - public List getAllCustomers() { - return customerService.getAll(); - } - - @POST - public Response createCustomer(Customer customer) { - var savedCustomer = customerService.create(customer); - return Response.status(Response.Status.CREATED).entity(savedCustomer).build(); - } -} -``` diff --git a/content/guides/testcontainers-java-quarkus/run-tests.md b/content/guides/testcontainers-java-quarkus/run-tests.md deleted file mode 100644 index 32525d7012d7..000000000000 --- a/content/guides/testcontainers-java-quarkus/run-tests.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -title: Run tests and next steps -linkTitle: Run tests -description: Run your Testcontainers-based Quarkus integration tests and explore next steps. -keywords: testcontainers, java, quarkus, postgresql, integration testing -weight: 30 ---- - -## Run the tests - -```console -$ ./mvnw test -``` - -Or with Gradle: - -```console -$ ./gradlew test -``` - -You should see the PostgreSQL Docker container start and all tests pass. After -the tests finish, the container stops and is removed automatically. - -## Run the application locally - -Quarkus Dev Services automatically provisions unconfigured services in -development mode. Start the Quarkus application in dev mode: - -```console -$ ./mvnw compile quarkus:dev -``` - -Or with Gradle: - -```console -$ ./gradlew quarkusDev -``` - -Dev Services starts a PostgreSQL container automatically. If you're running a -PostgreSQL database on your system and want to use that instead, configure the -datasource properties in `src/main/resources/application.properties`: - -```properties -quarkus.datasource.jdbc.url=jdbc:postgresql://localhost:5432/postgres -quarkus.datasource.username=postgres -quarkus.datasource.password=postgres -``` - -When these properties are set explicitly, Dev Services doesn't provision the -database container and instead connects to the configured database. - -## Summary - -Quarkus Dev Services improves the developer experience by automatically -provisioning the required services using Testcontainers during development and -testing. This guide covered: - -- Building a REST API using JAX-RS with Hibernate ORM with Panache -- Testing API endpoints using REST Assured with Dev Services handling database - provisioning -- Using `QuarkusTestResourceLifecycleManager` for services not supported by Dev - Services -- Running the application locally with Dev Services - -To learn more about Testcontainers, visit the -[Testcontainers overview](https://testcontainers.com/getting-started/). - -## Further reading - -- [Quarkus Dev Services overview](https://quarkus.io/guides/dev-services) -- [Quarkus testing guide](https://quarkus.io/guides/getting-started-testing) -- [Testcontainers Postgres module](https://java.testcontainers.org/modules/databases/postgres/) diff --git a/content/guides/testcontainers-java-quarkus/write-tests.md b/content/guides/testcontainers-java-quarkus/write-tests.md deleted file mode 100644 index a22ef561be26..000000000000 --- a/content/guides/testcontainers-java-quarkus/write-tests.md +++ /dev/null @@ -1,186 +0,0 @@ ---- -title: Write tests with Testcontainers -linkTitle: Write tests -description: Test the Quarkus REST API using Dev Services with Testcontainers, and test with services not supported by Dev Services. -keywords: testcontainers, java, quarkus, dev services, rest api, integration testing -weight: 20 ---- - -## Quarkus Dev Services - -Quarkus Dev Services automatically provisions unconfigured services in -development and test mode. When you include an extension and don't configure it, -Quarkus starts the relevant service using -[Testcontainers](https://www.testcontainers.org/) behind the scenes and wires -the application to use that service. - -> [!NOTE] -> Dev Services requires a -> [supported Docker environment](https://www.testcontainers.org/supported_docker_environment/). - -Quarkus Dev Services supports most commonly used services like SQL databases, -Kafka, RabbitMQ, Redis, and MongoDB. For more information, see the -[Quarkus Dev Services guide](https://quarkus.io/guides/dev-services). - -## Write tests for the API endpoints - -Test the `GET /api/customers` and `POST /api/customers` endpoints using REST -Assured. The `io.rest-assured:rest-assured` library was already added as a test -dependency when you generated the project. - -Create `CustomerResourceTest.java` and annotate it with `@QuarkusTest`. This -bootstraps the application along with the required services using Dev Services. -Because you haven't configured datasource properties, Dev Services automatically -starts a PostgreSQL database using Testcontainers. - -```java -package com.testcontainers.demo; - -import static io.restassured.RestAssured.given; -import static org.hamcrest.CoreMatchers.is; -import static org.junit.jupiter.api.Assertions.assertFalse; - -import io.quarkus.test.junit.QuarkusTest; -import io.restassured.common.mapper.TypeRef; -import io.restassured.http.ContentType; -import java.util.List; -import org.junit.jupiter.api.Test; - -@QuarkusTest -class CustomerResourceTest { - - @Test - void shouldGetAllCustomers() { - List customers = given().when() - .get("/api/customers") - .then() - .statusCode(200) - .extract() - .as(new TypeRef<>() {}); - assertFalse(customers.isEmpty()); - } - - @Test - void shouldCreateCustomerSuccessfully() { - Customer customer = new Customer(null, "John", "john@gmail.com"); - given().contentType(ContentType.JSON) - .body(customer) - .when() - .post("/api/customers") - .then() - .statusCode(201) - .body("name", is("John")) - .body("email", is("john@gmail.com")); - } -} -``` - -Here's what the test does: - -- `@QuarkusTest` starts the full Quarkus application with Dev Services enabled. -- Dev Services starts a PostgreSQL container using Testcontainers and configures - the datasource automatically. -- `shouldGetAllCustomers()` calls `GET /api/customers` and verifies that seeded - data from the Flyway migration is returned. -- `shouldCreateCustomerSuccessfully()` sends a `POST /api/customers` request and - verifies the response contains the created customer data. - -## Customize test configuration - -By default, the Quarkus test instance starts on port 8081 and uses a -`postgres:14` Docker image. Customize both by adding these properties to -`src/main/resources/application.properties`: - -```properties -quarkus.http.test-port=0 -quarkus.datasource.devservices.image-name=postgres:15.2-alpine -``` - -Setting `quarkus.http.test-port=0` starts the application on a random available -port, avoiding port conflicts. The `devservices.image-name` property lets you -pin the PostgreSQL image to a specific version that matches production. - -## Test with services not supported by Dev Services - -Your application might use a service that Dev Services doesn't support out of -the box. In that case, use `QuarkusTestResourceLifecycleManager` to start the -service before the Quarkus application starts for testing. - -For example, suppose the application uses CockroachDB. First, add the -CockroachDB Testcontainers module dependency: - -```xml - - org.testcontainers - cockroachdb - test - -``` - -Create a `CockroachDBTestResource` that implements -`QuarkusTestResourceLifecycleManager`: - -```java -package com.testcontainers.demo; - -import io.quarkus.test.common.QuarkusTestResourceLifecycleManager; -import java.util.HashMap; -import java.util.Map; -import org.testcontainers.containers.CockroachContainer; - -public class CockroachDBTestResource implements QuarkusTestResourceLifecycleManager { - - CockroachContainer cockroachdb; - - @Override - public Map start() { - cockroachdb = new CockroachContainer("cockroachdb/cockroach:v22.2.0"); - cockroachdb.start(); - Map conf = new HashMap<>(); - conf.put("quarkus.datasource.jdbc.url", cockroachdb.getJdbcUrl()); - conf.put("quarkus.datasource.username", cockroachdb.getUsername()); - conf.put("quarkus.datasource.password", cockroachdb.getPassword()); - return conf; - } - - @Override - public void stop() { - cockroachdb.stop(); - } -} -``` - -Use the `CockroachDBTestResource` with `@QuarkusTestResource` in a test class: - -```java -package com.testcontainers.demo; - -import static io.restassured.RestAssured.given; -import static org.junit.jupiter.api.Assertions.assertFalse; - -import io.quarkus.test.common.QuarkusTestResource; -import io.quarkus.test.junit.QuarkusTest; -import io.restassured.common.mapper.TypeRef; -import java.util.List; -import org.junit.jupiter.api.Test; - -@QuarkusTest -@QuarkusTestResource(value = CockroachDBTestResource.class, restrictToAnnotatedClass = true) -class CockroachDBTest { - - @Test - void shouldGetAllCustomers() { - List customers = given().when() - .get("/api/customers") - .then() - .statusCode(200) - .extract() - .as(new TypeRef<>() {}); - assertFalse(customers.isEmpty()); - } -} -``` - -The `restrictToAnnotatedClass = true` attribute ensures the CockroachDB -container only starts when running this specific test class, rather than being -activated for all tests. diff --git a/content/guides/testcontainers-java-replace-h2/_index.md b/content/guides/testcontainers-java-replace-h2/_index.md index c6dd22e70992..ee4062a99a83 100644 --- a/content/guides/testcontainers-java-replace-h2/_index.md +++ b/content/guides/testcontainers-java-replace-h2/_index.md @@ -6,14 +6,16 @@ keywords: testcontainers, java, testing, h2, postgresql, spring boot, spring dat summary: | Replace your H2 in-memory test database with a real PostgreSQL instance using the Testcontainers special JDBC URL — a one-line change. -toc_min: 1 -toc_max: 2 -tags: [testing-with-docker] -languages: [java] +aliases: + - /guides/testcontainers-java-replace-h2/jdbc-url-approach/ + - /guides/testcontainers-java-replace-h2/junit-extension-approach/ + - /guides/testcontainers-java-replace-h2/problem-with-h2/ params: + tags: [testing] time: 15 minutes --- + In this guide, you will learn how to: @@ -33,3 +35,232 @@ In this guide, you will learn how to: > If you're new to Testcontainers, visit the > [Testcontainers overview](https://testcontainers.com/getting-started/) to learn more about > Testcontainers and the benefits of using it. + +## The problem with H2 for testing + +A common practice is to use lightweight databases like H2 or HSQL as +in-memory databases for testing while using PostgreSQL, MySQL, or Oracle in +production. This approach has significant drawbacks: + +- The test database might not support all features of your production database. +- SQL syntax might not be compatible between H2 and your production database. +- Tests passing with H2 don't guarantee they'll work in production. + +### Example: PostgreSQL-specific syntax + +Consider implementing an "upsert" — insert a product only if it doesn't +already exist. In PostgreSQL, you can use: + +```sql +INSERT INTO products(id, code, name) VALUES(?,?,?) ON CONFLICT DO NOTHING; +``` + +This query doesn't work with H2 by default: + +```text +Caused by: org.h2.jdbc.JdbcSQLException: Syntax error in SQL statement +"INSERT INTO products (id, code, name) VALUES (?, ?, ?) ON[*] CONFLICT DO NOTHING"; +``` + +You can run H2 in PostgreSQL compatibility mode, but not all features are +supported. The inverse is also true — H2 supports `ROWNUM()` which PostgreSQL +doesn't. + +Testing with a different database than production means you can't trust your +test results and must verify after deployment, defeating the purpose of +automated tests. + +### The Spring Boot test using H2 + +A typical H2-based test looks like this: + +```java +@DataJpaTest +class ProductRepositoryTest { + + @Autowired + ProductRepository productRepository; + + @Test + @Sql("classpath:/sql/seed-data.sql") + void shouldGetAllProducts() { + List products = productRepository.findAll(); + assertEquals(2, products.size()); + } +} +``` + +Spring Boot uses H2 automatically when it's on the classpath. The test passes, +but it doesn't catch PostgreSQL-specific issues. + +## Replace H2 with the Testcontainers JDBC URL + +Replacing H2 with a real PostgreSQL database requires two test properties: + +```java +@DataJpaTest +@TestPropertySource(properties = { + "spring.test.database.replace=none", + "spring.datasource.url=jdbc:tc:postgresql:16-alpine:///db" +}) +class ProductRepositoryWithJdbcUrlTest { + + @Autowired + ProductRepository productRepository; + + @Test + @Sql("classpath:/sql/seed-data.sql") + void shouldGetAllProducts() { + List products = productRepository.findAll(); + assertEquals(2, products.size()); + } +} +``` + +That's it — two properties and your tests run against a real PostgreSQL +database. + +### How the special JDBC URL works + +A standard PostgreSQL JDBC URL looks like: + +```text +jdbc:postgresql://localhost:5432/postgres +``` + +The Testcontainers special JDBC URL inserts `tc:` after `jdbc:`: + +```text +jdbc:tc:postgresql:///db +``` + +The hostname, port, and database name are ignored — Testcontainers manages them +automatically. You can specify the Docker image tag after the database name: + +```text +jdbc:tc:postgresql:16-alpine:///db +``` + +This creates a container from the `postgres:16-alpine` image. + +### Initialize the database with a script + +Pass `TC_INITSCRIPT` to run an SQL script when the container starts: + +```text +jdbc:tc:postgresql:16-alpine:///db?TC_INITSCRIPT=sql/init-db.sql +``` + +Testcontainers runs the script automatically. For production applications, +use a database migration tool like Flyway or Liquibase instead. + +The special JDBC URL also works for MySQL, MariaDB, PostGIS, YugabyteDB, +CockroachDB, and other databases with Testcontainers JDBC support. + +### Testing JdbcTemplate-based repositories + +The same approach works for `JdbcTemplate`-based repositories. Use `@JdbcTest` +instead of `@DataJpaTest`: + +```java +@JdbcTest +@TestPropertySource(properties = { + "spring.test.database.replace=none", + "spring.datasource.url=jdbc:tc:postgresql:16-alpine:///db?TC_INITSCRIPT=sql/init-db.sql" +}) +class JdbcProductRepositoryTest { + + @Autowired + private JdbcTemplate jdbcTemplate; + + private JdbcProductRepository productRepository; + + @BeforeEach + void setUp() { + productRepository = new JdbcProductRepository(jdbcTemplate); + } + + @Test + @Sql("/sql/seed-data.sql") + void shouldGetAllProducts() { + List products = productRepository.getAllProducts(); + assertEquals(2, products.size()); + } +} +``` + +## Use the JUnit 5 extension for more control + +If the special JDBC URL doesn't meet your needs, or you need more control over +container creation (for example, to copy initialization scripts), use the +Testcontainers JUnit 5 extension: + +```java +@DataJpaTest +@TestPropertySource(properties = { + "spring.test.database.replace=none" +}) +@Testcontainers +class ProductRepositoryTest { + + @Container + static PostgreSQLContainer postgres = + new PostgreSQLContainer("postgres:16-alpine") + .withCopyFileToContainer( + MountableFile.forClasspathResource("sql/init-db.sql"), + "/docker-entrypoint-initdb.d/init-db.sql"); + + @DynamicPropertySource + static void configureProperties(DynamicPropertyRegistry registry) { + registry.add("spring.datasource.url", postgres::getJdbcUrl); + registry.add("spring.datasource.username", postgres::getUsername); + registry.add("spring.datasource.password", postgres::getPassword); + } + + @Autowired + ProductRepository productRepository; + + @Test + @Sql("/sql/seed-data.sql") + void shouldGetAllProducts() { + List products = productRepository.findAll(); + assertEquals(2, products.size()); + } + + @Test + @Sql("/sql/seed-data.sql") + void shouldNotCreateAProductWithDuplicateCode() { + Product product = new Product(3L, "p101", "Test Product"); + productRepository.createProductIfNotExists(product); + Optional optionalProduct = productRepository.findById( + product.getId() + ); + assertThat(optionalProduct).isEmpty(); + } +} +``` + +This approach: + +- Uses `@Testcontainers` and `@Container` to manage the container lifecycle. +- Copies `init-db.sql` into the container's init directory so PostgreSQL + runs it at startup. +- Uses `@DynamicPropertySource` to register the container's connection details + with Spring Boot. +- Tests PostgreSQL-specific features like `ON CONFLICT DO NOTHING` that + wouldn't work with H2. + +### Summary + +- Use the **special JDBC URL** (`jdbc:tc:postgresql:...`) for the quickest way + to switch from H2 to a real database — it's a one-property change. +- Use the **JUnit 5 extension** when you need more control over the container + (custom init scripts, environment variables, etc.). +- Both approaches work with Spring Data JPA (`@DataJpaTest`) and JdbcTemplate + (`@JdbcTest`) tests. + +### Further reading + +- [Testcontainers Postgres module](https://java.testcontainers.org/modules/databases/postgres/) +- [Testcontainers JDBC support](https://java.testcontainers.org/modules/databases/jdbc/) +- [Testing a Spring Boot REST API with Testcontainers](/guides/testcontainers-java-spring-boot-rest-api/) diff --git a/content/guides/testcontainers-java-replace-h2/jdbc-url-approach.md b/content/guides/testcontainers-java-replace-h2/jdbc-url-approach.md deleted file mode 100644 index a09dcaf23051..000000000000 --- a/content/guides/testcontainers-java-replace-h2/jdbc-url-approach.md +++ /dev/null @@ -1,101 +0,0 @@ ---- -title: Replace H2 with the Testcontainers JDBC URL -linkTitle: JDBC URL approach -description: Use the Testcontainers special JDBC URL to swap H2 for a real PostgreSQL database. -keywords: testcontainers, java, h2, postgresql, jdbc, integration testing -weight: 20 ---- - -Replacing H2 with a real PostgreSQL database requires two test properties: - -```java -@DataJpaTest -@TestPropertySource(properties = { - "spring.test.database.replace=none", - "spring.datasource.url=jdbc:tc:postgresql:16-alpine:///db" -}) -class ProductRepositoryWithJdbcUrlTest { - - @Autowired - ProductRepository productRepository; - - @Test - @Sql("classpath:/sql/seed-data.sql") - void shouldGetAllProducts() { - List products = productRepository.findAll(); - assertEquals(2, products.size()); - } -} -``` - -That's it — two properties and your tests run against a real PostgreSQL -database. - -## How the special JDBC URL works - -A standard PostgreSQL JDBC URL looks like: - -```text -jdbc:postgresql://localhost:5432/postgres -``` - -The Testcontainers special JDBC URL inserts `tc:` after `jdbc:`: - -```text -jdbc:tc:postgresql:///db -``` - -The hostname, port, and database name are ignored — Testcontainers manages them -automatically. You can specify the Docker image tag after the database name: - -```text -jdbc:tc:postgresql:16-alpine:///db -``` - -This creates a container from the `postgres:16-alpine` image. - -## Initialize the database with a script - -Pass `TC_INITSCRIPT` to run an SQL script when the container starts: - -```text -jdbc:tc:postgresql:16-alpine:///db?TC_INITSCRIPT=sql/init-db.sql -``` - -Testcontainers runs the script automatically. For production applications, -use a database migration tool like Flyway or Liquibase instead. - -The special JDBC URL also works for MySQL, MariaDB, PostGIS, YugabyteDB, -CockroachDB, and other databases with Testcontainers JDBC support. - -## Testing JdbcTemplate-based repositories - -The same approach works for `JdbcTemplate`-based repositories. Use `@JdbcTest` -instead of `@DataJpaTest`: - -```java -@JdbcTest -@TestPropertySource(properties = { - "spring.test.database.replace=none", - "spring.datasource.url=jdbc:tc:postgresql:16-alpine:///db?TC_INITSCRIPT=sql/init-db.sql" -}) -class JdbcProductRepositoryTest { - - @Autowired - private JdbcTemplate jdbcTemplate; - - private JdbcProductRepository productRepository; - - @BeforeEach - void setUp() { - productRepository = new JdbcProductRepository(jdbcTemplate); - } - - @Test - @Sql("/sql/seed-data.sql") - void shouldGetAllProducts() { - List products = productRepository.getAllProducts(); - assertEquals(2, products.size()); - } -} -``` diff --git a/content/guides/testcontainers-java-replace-h2/junit-extension-approach.md b/content/guides/testcontainers-java-replace-h2/junit-extension-approach.md deleted file mode 100644 index 75c11b516685..000000000000 --- a/content/guides/testcontainers-java-replace-h2/junit-extension-approach.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -title: Use the JUnit 5 extension for more control -linkTitle: JUnit 5 extension -description: Use the Testcontainers JUnit 5 extension for more control over the PostgreSQL container. -keywords: testcontainers, java, h2, postgresql, junit, integration testing -weight: 30 ---- - -If the special JDBC URL doesn't meet your needs, or you need more control over -container creation (for example, to copy initialization scripts), use the -Testcontainers JUnit 5 extension: - -```java -@DataJpaTest -@TestPropertySource(properties = { - "spring.test.database.replace=none" -}) -@Testcontainers -class ProductRepositoryTest { - - @Container - static PostgreSQLContainer postgres = - new PostgreSQLContainer("postgres:16-alpine") - .withCopyFileToContainer( - MountableFile.forClasspathResource("sql/init-db.sql"), - "/docker-entrypoint-initdb.d/init-db.sql"); - - @DynamicPropertySource - static void configureProperties(DynamicPropertyRegistry registry) { - registry.add("spring.datasource.url", postgres::getJdbcUrl); - registry.add("spring.datasource.username", postgres::getUsername); - registry.add("spring.datasource.password", postgres::getPassword); - } - - @Autowired - ProductRepository productRepository; - - @Test - @Sql("/sql/seed-data.sql") - void shouldGetAllProducts() { - List products = productRepository.findAll(); - assertEquals(2, products.size()); - } - - @Test - @Sql("/sql/seed-data.sql") - void shouldNotCreateAProductWithDuplicateCode() { - Product product = new Product(3L, "p101", "Test Product"); - productRepository.createProductIfNotExists(product); - Optional optionalProduct = productRepository.findById( - product.getId() - ); - assertThat(optionalProduct).isEmpty(); - } -} -``` - -This approach: - -- Uses `@Testcontainers` and `@Container` to manage the container lifecycle. -- Copies `init-db.sql` into the container's init directory so PostgreSQL - runs it at startup. -- Uses `@DynamicPropertySource` to register the container's connection details - with Spring Boot. -- Tests PostgreSQL-specific features like `ON CONFLICT DO NOTHING` that - wouldn't work with H2. - -## Summary - -- Use the **special JDBC URL** (`jdbc:tc:postgresql:...`) for the quickest way - to switch from H2 to a real database — it's a one-property change. -- Use the **JUnit 5 extension** when you need more control over the container - (custom init scripts, environment variables, etc.). -- Both approaches work with Spring Data JPA (`@DataJpaTest`) and JdbcTemplate - (`@JdbcTest`) tests. - -## Further reading - -- [Testcontainers Postgres module](https://java.testcontainers.org/modules/databases/postgres/) -- [Testcontainers JDBC support](https://java.testcontainers.org/modules/databases/jdbc/) -- [Testing a Spring Boot REST API with Testcontainers](/guides/testcontainers-java-spring-boot-rest-api/) diff --git a/content/guides/testcontainers-java-replace-h2/problem-with-h2.md b/content/guides/testcontainers-java-replace-h2/problem-with-h2.md deleted file mode 100644 index c51c05961a8f..000000000000 --- a/content/guides/testcontainers-java-replace-h2/problem-with-h2.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -title: The problem with H2 for testing -linkTitle: The H2 problem -description: Understand why using H2 in-memory databases for testing gives false confidence. -keywords: testcontainers, java, h2, in-memory database, integration testing -weight: 10 ---- - -A common practice is to use lightweight databases like H2 or HSQL as -in-memory databases for testing while using PostgreSQL, MySQL, or Oracle in -production. This approach has significant drawbacks: - -- The test database might not support all features of your production database. -- SQL syntax might not be compatible between H2 and your production database. -- Tests passing with H2 don't guarantee they'll work in production. - -## Example: PostgreSQL-specific syntax - -Consider implementing an "upsert" — insert a product only if it doesn't -already exist. In PostgreSQL, you can use: - -```sql -INSERT INTO products(id, code, name) VALUES(?,?,?) ON CONFLICT DO NOTHING; -``` - -This query doesn't work with H2 by default: - -```text -Caused by: org.h2.jdbc.JdbcSQLException: Syntax error in SQL statement -"INSERT INTO products (id, code, name) VALUES (?, ?, ?) ON[*] CONFLICT DO NOTHING"; -``` - -You can run H2 in PostgreSQL compatibility mode, but not all features are -supported. The inverse is also true — H2 supports `ROWNUM()` which PostgreSQL -doesn't. - -Testing with a different database than production means you can't trust your -test results and must verify after deployment, defeating the purpose of -automated tests. - -## The Spring Boot test using H2 - -A typical H2-based test looks like this: - -```java -@DataJpaTest -class ProductRepositoryTest { - - @Autowired - ProductRepository productRepository; - - @Test - @Sql("classpath:/sql/seed-data.sql") - void shouldGetAllProducts() { - List products = productRepository.findAll(); - assertEquals(2, products.size()); - } -} -``` - -Spring Boot uses H2 automatically when it's on the classpath. The test passes, -but it doesn't catch PostgreSQL-specific issues. diff --git a/content/guides/testcontainers-java-service-configuration/_index.md b/content/guides/testcontainers-java-service-configuration/_index.md index 2d1ac1d788c6..c969f11b2bee 100644 --- a/content/guides/testcontainers-java-service-configuration/_index.md +++ b/content/guides/testcontainers-java-service-configuration/_index.md @@ -6,14 +6,15 @@ keywords: testcontainers, java, testing, postgresql, localstack, container confi summary: | Learn how to initialize and configure Docker containers for testing by copying files into containers and executing commands inside them. -toc_min: 1 -toc_max: 2 -tags: [testing-with-docker] -languages: [java] +aliases: + - /guides/testcontainers-java-service-configuration/copy-files/ + - /guides/testcontainers-java-service-configuration/exec-in-container/ params: + tags: [testing] time: 15 minutes --- + In this guide, you will learn how to: @@ -33,3 +34,200 @@ In this guide, you will learn how to: > If you're new to Testcontainers, visit the > [Testcontainers overview](https://testcontainers.com/getting-started/) to learn more about > Testcontainers and the benefits of using it. + +## Copy files into containers + +Sometimes you need to initialize a container by placing files in a specific +location. For example, PostgreSQL runs SQL scripts from +`/docker-entrypoint-initdb.d/` when the container starts. + +### Create the initialization script + +Create `src/test/resources/init-db.sql`: + +```sql +create table customers ( + id bigint not null, + name varchar not null, + primary key (id) +); +``` + +### Copy the file into the container + +Use `withCopyFileToContainer()` to copy the SQL script into the container's +init directory: + +```java +package com.testcontainers.demo; + +import static org.junit.jupiter.api.Assertions.assertFalse; + +import java.util.List; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.testcontainers.postgresql.PostgreSQLContainer; +import org.testcontainers.junit.jupiter.Container; +import org.testcontainers.junit.jupiter.Testcontainers; +import org.testcontainers.utility.MountableFile; + +@Testcontainers +class CustomerServiceTest { + + @Container + static PostgreSQLContainer postgres = new PostgreSQLContainer( + "postgres:16-alpine" + ) + .withCopyFileToContainer( + MountableFile.forClasspathResource("init-db.sql"), + "/docker-entrypoint-initdb.d/" + ); + + CustomerService customerService; + + @BeforeEach + void setUp() { + customerService = + new CustomerService( + postgres.getJdbcUrl(), + postgres.getUsername(), + postgres.getPassword() + ); + } + + @Test + void shouldGetCustomers() { + customerService.createCustomer(new Customer(1L, "George")); + customerService.createCustomer(new Customer(2L, "John")); + + List customers = customerService.getAllCustomers(); + assertFalse(customers.isEmpty()); + } +} +``` + +The `withCopyFileToContainer(MountableFile, String)` method copies `init-db.sql` +from the classpath into `/docker-entrypoint-initdb.d/` inside the container. +PostgreSQL executes scripts in that directory automatically at startup. + +You can also copy files from any path on the host: + +```java +.withCopyFileToContainer( + MountableFile.forHostPath("/host/path/to/init-db.sql"), + "/docker-entrypoint-initdb.d/" +); +``` + +## Execute commands inside containers + +Some Docker containers provide CLI tools for performing actions. You can use +`container.execInContainer(String...)` to run any available command inside a +running container. + +### Example: Create an S3 bucket in LocalStack + +The [LocalStack](https://localstack.cloud/) module emulates AWS services. To +test S3 file uploads, create a bucket before running tests: + +```java +package com.testcontainers.demo; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.testcontainers.containers.localstack.LocalStackContainer.Service.S3; + +import java.io.IOException; +import java.net.URI; +import java.util.List; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; +import org.testcontainers.containers.localstack.LocalStackContainer; +import org.testcontainers.junit.jupiter.Container; +import org.testcontainers.junit.jupiter.Testcontainers; +import org.testcontainers.utility.DockerImageName; +import software.amazon.awssdk.auth.credentials.AwsBasicCredentials; +import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider; +import software.amazon.awssdk.regions.Region; +import software.amazon.awssdk.services.s3.S3Client; +import software.amazon.awssdk.services.s3.model.Bucket; + +@Testcontainers +class LocalStackTest { + + static final String bucketName = "mybucket"; + + @Container + static LocalStackContainer localStack = new LocalStackContainer( + DockerImageName.parse("localstack/localstack:3.4.0") + ); + + @BeforeAll + static void beforeAll() throws IOException, InterruptedException { + localStack.execInContainer("awslocal", "s3", "mb", "s3://" + bucketName); + + org.testcontainers.containers.Container.ExecResult execResult = + localStack.execInContainer("awslocal", "s3", "ls"); + String stdout = execResult.getStdout(); + int exitCode = execResult.getExitCode(); + assertTrue(stdout.contains(bucketName)); + assertEquals(0, exitCode); + } + + @Test + void shouldListBuckets() { + URI s3Endpoint = localStack.getEndpointOverride(S3); + StaticCredentialsProvider credentialsProvider = + StaticCredentialsProvider.create( + AwsBasicCredentials.create( + localStack.getAccessKey(), + localStack.getSecretKey() + ) + ); + S3Client s3 = S3Client + .builder() + .endpointOverride(s3Endpoint) + .credentialsProvider(credentialsProvider) + .region(Region.of(localStack.getRegion())) + .build(); + + List s3Buckets = s3 + .listBuckets() + .buckets() + .stream() + .map(Bucket::name) + .toList(); + + assertTrue(s3Buckets.contains(bucketName)); + } +} +``` + +The `execInContainer("awslocal", "s3", "mb", "s3://mybucket")` call runs the +`awslocal` CLI tool (provided by the LocalStack image) to create an S3 bucket. + +You can capture the output and exit code from any command: + +```java +Container.ExecResult execResult = + localStack.execInContainer("awslocal", "s3", "ls"); +String stdout = execResult.getStdout(); +int exitCode = execResult.getExitCode(); +``` + +> [!NOTE] +> The `withCopyFileToContainer()` and `execInContainer()` methods are inherited +> from `GenericContainer`, so they're available for all Testcontainers modules. + +### Summary + +- Use `withCopyFileToContainer()` to place initialization files inside + containers before they start. +- Use `execInContainer()` to run commands inside running containers for + setup tasks like creating buckets, topics, or queues. + +### Further reading + +- [Getting started with Testcontainers for Java](/guides/testcontainers-java-getting-started/) +- [Testcontainers Postgres module](https://java.testcontainers.org/modules/databases/postgres/) +- [Testcontainers LocalStack module](https://java.testcontainers.org/modules/localstack/) diff --git a/content/guides/testcontainers-java-service-configuration/copy-files.md b/content/guides/testcontainers-java-service-configuration/copy-files.md deleted file mode 100644 index 7a992bfc4c8b..000000000000 --- a/content/guides/testcontainers-java-service-configuration/copy-files.md +++ /dev/null @@ -1,89 +0,0 @@ ---- -title: Copy files into containers -linkTitle: Copy files -description: Initialize containers by copying files into specific locations. -keywords: testcontainers, java, service configuration, copy files, container initialization -weight: 10 ---- - -Sometimes you need to initialize a container by placing files in a specific -location. For example, PostgreSQL runs SQL scripts from -`/docker-entrypoint-initdb.d/` when the container starts. - -## Create the initialization script - -Create `src/test/resources/init-db.sql`: - -```sql -create table customers ( - id bigint not null, - name varchar not null, - primary key (id) -); -``` - -## Copy the file into the container - -Use `withCopyFileToContainer()` to copy the SQL script into the container's -init directory: - -```java -package com.testcontainers.demo; - -import static org.junit.jupiter.api.Assertions.assertFalse; - -import java.util.List; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.Test; -import org.testcontainers.postgresql.PostgreSQLContainer; -import org.testcontainers.junit.jupiter.Container; -import org.testcontainers.junit.jupiter.Testcontainers; -import org.testcontainers.utility.MountableFile; - -@Testcontainers -class CustomerServiceTest { - - @Container - static PostgreSQLContainer postgres = new PostgreSQLContainer( - "postgres:16-alpine" - ) - .withCopyFileToContainer( - MountableFile.forClasspathResource("init-db.sql"), - "/docker-entrypoint-initdb.d/" - ); - - CustomerService customerService; - - @BeforeEach - void setUp() { - customerService = - new CustomerService( - postgres.getJdbcUrl(), - postgres.getUsername(), - postgres.getPassword() - ); - } - - @Test - void shouldGetCustomers() { - customerService.createCustomer(new Customer(1L, "George")); - customerService.createCustomer(new Customer(2L, "John")); - - List customers = customerService.getAllCustomers(); - assertFalse(customers.isEmpty()); - } -} -``` - -The `withCopyFileToContainer(MountableFile, String)` method copies `init-db.sql` -from the classpath into `/docker-entrypoint-initdb.d/` inside the container. -PostgreSQL executes scripts in that directory automatically at startup. - -You can also copy files from any path on the host: - -```java -.withCopyFileToContainer( - MountableFile.forHostPath("/host/path/to/init-db.sql"), - "/docker-entrypoint-initdb.d/" -); -``` diff --git a/content/guides/testcontainers-java-service-configuration/exec-in-container.md b/content/guides/testcontainers-java-service-configuration/exec-in-container.md deleted file mode 100644 index 0ac4dc1ee01f..000000000000 --- a/content/guides/testcontainers-java-service-configuration/exec-in-container.md +++ /dev/null @@ -1,118 +0,0 @@ ---- -title: Execute commands inside containers -linkTitle: Execute commands -description: Run commands inside running containers to initialize services for testing. -keywords: testcontainers, java, service configuration, exec, container initialization -weight: 20 ---- - -Some Docker containers provide CLI tools for performing actions. You can use -`container.execInContainer(String...)` to run any available command inside a -running container. - -## Example: Create an S3 bucket in LocalStack - -The [LocalStack](https://localstack.cloud/) module emulates AWS services. To -test S3 file uploads, create a bucket before running tests: - -```java -package com.testcontainers.demo; - -import static org.junit.jupiter.api.Assertions.assertEquals; -import static org.junit.jupiter.api.Assertions.assertTrue; -import static org.testcontainers.containers.localstack.LocalStackContainer.Service.S3; - -import java.io.IOException; -import java.net.URI; -import java.util.List; -import org.junit.jupiter.api.BeforeAll; -import org.junit.jupiter.api.Test; -import org.testcontainers.containers.localstack.LocalStackContainer; -import org.testcontainers.junit.jupiter.Container; -import org.testcontainers.junit.jupiter.Testcontainers; -import org.testcontainers.utility.DockerImageName; -import software.amazon.awssdk.auth.credentials.AwsBasicCredentials; -import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider; -import software.amazon.awssdk.regions.Region; -import software.amazon.awssdk.services.s3.S3Client; -import software.amazon.awssdk.services.s3.model.Bucket; - -@Testcontainers -class LocalStackTest { - - static final String bucketName = "mybucket"; - - @Container - static LocalStackContainer localStack = new LocalStackContainer( - DockerImageName.parse("localstack/localstack:3.4.0") - ); - - @BeforeAll - static void beforeAll() throws IOException, InterruptedException { - localStack.execInContainer("awslocal", "s3", "mb", "s3://" + bucketName); - - org.testcontainers.containers.Container.ExecResult execResult = - localStack.execInContainer("awslocal", "s3", "ls"); - String stdout = execResult.getStdout(); - int exitCode = execResult.getExitCode(); - assertTrue(stdout.contains(bucketName)); - assertEquals(0, exitCode); - } - - @Test - void shouldListBuckets() { - URI s3Endpoint = localStack.getEndpointOverride(S3); - StaticCredentialsProvider credentialsProvider = - StaticCredentialsProvider.create( - AwsBasicCredentials.create( - localStack.getAccessKey(), - localStack.getSecretKey() - ) - ); - S3Client s3 = S3Client - .builder() - .endpointOverride(s3Endpoint) - .credentialsProvider(credentialsProvider) - .region(Region.of(localStack.getRegion())) - .build(); - - List s3Buckets = s3 - .listBuckets() - .buckets() - .stream() - .map(Bucket::name) - .toList(); - - assertTrue(s3Buckets.contains(bucketName)); - } -} -``` - -The `execInContainer("awslocal", "s3", "mb", "s3://mybucket")` call runs the -`awslocal` CLI tool (provided by the LocalStack image) to create an S3 bucket. - -You can capture the output and exit code from any command: - -```java -Container.ExecResult execResult = - localStack.execInContainer("awslocal", "s3", "ls"); -String stdout = execResult.getStdout(); -int exitCode = execResult.getExitCode(); -``` - -> [!NOTE] -> The `withCopyFileToContainer()` and `execInContainer()` methods are inherited -> from `GenericContainer`, so they're available for all Testcontainers modules. - -## Summary - -- Use `withCopyFileToContainer()` to place initialization files inside - containers before they start. -- Use `execInContainer()` to run commands inside running containers for - setup tasks like creating buckets, topics, or queues. - -## Further reading - -- [Getting started with Testcontainers for Java](/guides/testcontainers-java-getting-started/) -- [Testcontainers Postgres module](https://java.testcontainers.org/modules/databases/postgres/) -- [Testcontainers LocalStack module](https://java.testcontainers.org/modules/localstack/) diff --git a/content/guides/testcontainers-java-spring-boot-kafka/_index.md b/content/guides/testcontainers-java-spring-boot-kafka/_index.md index 03ff02d617e0..203db79898f8 100644 --- a/content/guides/testcontainers-java-spring-boot-kafka/_index.md +++ b/content/guides/testcontainers-java-spring-boot-kafka/_index.md @@ -6,14 +6,16 @@ keywords: testcontainers, java, spring boot, testing, kafka, mysql, jpa, awaitil summary: | Learn how to create a Spring Boot application with a Kafka listener that persists data in MySQL, then test it using Testcontainers Kafka and MySQL modules with Awaitility. -toc_min: 1 -toc_max: 2 -tags: [testing-with-docker] -languages: [java] +aliases: + - /guides/testcontainers-java-spring-boot-kafka/create-project/ + - /guides/testcontainers-java-spring-boot-kafka/run-tests/ + - /guides/testcontainers-java-spring-boot-kafka/write-tests/ params: + tags: [testing] time: 25 minutes --- + In this guide, you will learn how to: @@ -32,3 +34,438 @@ In this guide, you will learn how to: > If you're new to Testcontainers, visit the > [Testcontainers overview](https://testcontainers.com/getting-started/) to learn more about > Testcontainers and the benefits of using it. + +## Create the Spring Boot project + +### Set up the project + +Create a Spring Boot project from [Spring Initializr](https://start.spring.io) +by selecting the **Spring for Apache Kafka**, **Spring Data JPA**, **MySQL Driver**, +and **Testcontainers** starters. + +Alternatively, clone the +[guide repository](https://github.com/testcontainers/tc-guide-testing-spring-boot-kafka-listener). + +After generating the application, add the Awaitility library as a test +dependency. You'll use it later to assert the expectations of an asynchronous +process flow. + +The key dependencies in `pom.xml` are: + +```xml + + 17 + 2.0.4 + + + + org.springframework.boot + spring-boot-starter-data-jpa + + + org.springframework.kafka + spring-kafka + + + com.mysql + mysql-connector-j + runtime + + + org.springframework.boot + spring-boot-starter-test + test + + + org.springframework.kafka + spring-kafka-test + test + + + org.testcontainers + testcontainers-junit-jupiter + test + + + org.testcontainers + testcontainers-kafka + test + + + org.testcontainers + testcontainers-mysql + test + + + org.awaitility + awaitility + test + + +``` + +Using the Testcontainers BOM (Bill of Materials) is recommended so that you +don't have to repeat the version for every Testcontainers module dependency. + +### Create the JPA entity + +The application listens to a topic called `product-price-changes`. When a +message arrives, it extracts the product code and price from the event payload +and updates the price for that product in the MySQL database. + +Create `Product.java`: + +```java +package com.testcontainers.demo; + +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.GeneratedValue; +import jakarta.persistence.GenerationType; +import jakarta.persistence.Id; +import jakarta.persistence.Table; +import java.math.BigDecimal; + +@Entity +@Table(name = "products") +class Product { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(nullable = false, unique = true) + private String code; + + @Column(nullable = false) + private String name; + + @Column(nullable = false) + private BigDecimal price; + + public Product() {} + + public Product(Long id, String code, String name, BigDecimal price) { + this.id = id; + this.code = code; + this.name = name; + this.price = price; + } + + public Long getId() { + return id; + } + + public void setId(Long id) { + this.id = id; + } + + public String getCode() { + return code; + } + + public void setCode(String code) { + this.code = code; + } + + public String getName() { + return name; + } + + public void setName(String name) { + this.name = name; + } + + public BigDecimal getPrice() { + return price; + } + + public void setPrice(BigDecimal price) { + this.price = price; + } +} +``` + +### Create the Spring Data JPA repository + +Create a repository interface for the `Product` entity with a method to find a +product by code and a method to update the price for a given product code: + +```java +package com.testcontainers.demo; + +import java.math.BigDecimal; +import java.util.Optional; +import org.springframework.data.jpa.repository.JpaRepository; +import org.springframework.data.jpa.repository.Modifying; +import org.springframework.data.jpa.repository.Query; +import org.springframework.data.repository.query.Param; + +interface ProductRepository extends JpaRepository { + Optional findByCode(String code); + + @Modifying + @Query("update Product p set p.price = :price where p.code = :productCode") + void updateProductPrice( + @Param("productCode") String productCode, + @Param("price") BigDecimal price + ); +} +``` + +### Add a schema creation script + +Because the application doesn't use an in-memory database, you need to create +the MySQL tables. The recommended approach for production is a migration tool +like Flyway or Liquibase, but for this guide the built-in Spring Boot schema +initialization is sufficient. + +Create `src/main/resources/schema.sql`: + +```sql +create table products ( + id int NOT NULL AUTO_INCREMENT, + code varchar(255) not null, + name varchar(255) not null, + price numeric(5,2) not null, + PRIMARY KEY (id), + UNIQUE (code) +); +``` + +Enable schema initialization in `src/main/resources/application.properties`: + +```properties +spring.sql.init.mode=always +``` + +### Create the event payload + +Create a record named `ProductPriceChangedEvent` that represents the structure +of the event payload received from the Kafka topic: + +```java +package com.testcontainers.demo; + +import java.math.BigDecimal; + +record ProductPriceChangedEvent(String productCode, BigDecimal price) {} +``` + +The sender and receiver agree on the following JSON format: + +```json +{ + "productCode": "P100", + "price": 25.0 +} +``` + +### Implement the Kafka listener + +Create `ProductPriceChangedEventHandler.java`, which handles messages from the +`product-price-changes` topic and updates the product price in the database: + +```java +package com.testcontainers.demo; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.kafka.annotation.KafkaListener; +import org.springframework.stereotype.Component; +import org.springframework.transaction.annotation.Transactional; + +@Component +@Transactional +class ProductPriceChangedEventHandler { + + private static final Logger log = LoggerFactory.getLogger( + ProductPriceChangedEventHandler.class + ); + + private final ProductRepository productRepository; + + ProductPriceChangedEventHandler(ProductRepository productRepository) { + this.productRepository = productRepository; + } + + @KafkaListener(topics = "product-price-changes", groupId = "demo") + public void handle(ProductPriceChangedEvent event) { + log.info( + "Received a ProductPriceChangedEvent with productCode:{}: ", + event.productCode() + ); + productRepository.updateProductPrice(event.productCode(), event.price()); + } +} +``` + +The `@KafkaListener` annotation specifies the topic name to listen to. Spring +Kafka handles serialization and deserialization based on the properties +configured in `application.properties`. + +### Configure Kafka serialization + +Add the following Kafka properties to +`src/main/resources/application.properties`: + +```properties +######## Kafka Configuration ######### +spring.kafka.bootstrap-servers=localhost:9092 +spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer +spring.kafka.producer.value-serializer=org.springframework.kafka.support.serializer.JsonSerializer + +spring.kafka.consumer.group-id=demo +spring.kafka.consumer.auto-offset-reset=latest +spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer +spring.kafka.consumer.value-deserializer=org.springframework.kafka.support.serializer.JsonDeserializer +spring.kafka.consumer.properties.spring.json.trusted.packages=com.testcontainers.demo +``` + +The `productCode` key is (de)serialized using `StringSerializer`/`StringDeserializer`, +and the `ProductPriceChangedEvent` value is (de)serialized using +`JsonSerializer`/`JsonDeserializer`. + +## Write tests with Testcontainers + +To test the Kafka listener, you need a running Kafka broker and a MySQL +database, plus a started Spring context. Testcontainers spins up both services +in Docker containers and `@DynamicPropertySource` connects them to Spring. + +### Write the test + +Create `ProductPriceChangedEventHandlerTest.java`: + +```java +package com.testcontainers.demo; + +import static java.util.concurrent.TimeUnit.SECONDS; +import static org.assertj.core.api.Assertions.assertThat; +import static org.awaitility.Awaitility.await; + +import java.math.BigDecimal; +import java.time.Duration; +import java.util.Optional; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.kafka.core.KafkaTemplate; +import org.springframework.test.context.DynamicPropertyRegistry; +import org.springframework.test.context.DynamicPropertySource; +import org.springframework.test.context.TestPropertySource; +import org.testcontainers.kafka.ConfluentKafkaContainer; +import org.testcontainers.junit.jupiter.Container; +import org.testcontainers.junit.jupiter.Testcontainers; + +@SpringBootTest +@TestPropertySource( + properties = { + "spring.kafka.consumer.auto-offset-reset=earliest", + "spring.datasource.url=jdbc:tc:mysql:8.0.32:///db", + } +) +@Testcontainers +class ProductPriceChangedEventHandlerTest { + + @Container + static final ConfluentKafkaContainer kafka = + new ConfluentKafkaContainer("confluentinc/cp-kafka:7.8.0"); + + @DynamicPropertySource + static void overrideProperties(DynamicPropertyRegistry registry) { + registry.add("spring.kafka.bootstrap-servers", kafka::getBootstrapServers); + } + + @Autowired + private KafkaTemplate kafkaTemplate; + + @Autowired + private ProductRepository productRepository; + + @BeforeEach + void setUp() { + Product product = new Product(null, "P100", "Product One", BigDecimal.TEN); + productRepository.save(product); + } + + @Test + void shouldHandleProductPriceChangedEvent() { + ProductPriceChangedEvent event = new ProductPriceChangedEvent( + "P100", + new BigDecimal("14.50") + ); + + kafkaTemplate.send("product-price-changes", event.productCode(), event); + + await() + .pollInterval(Duration.ofSeconds(3)) + .atMost(10, SECONDS) + .untilAsserted(() -> { + Optional optionalProduct = productRepository.findByCode( + "P100" + ); + assertThat(optionalProduct).isPresent(); + assertThat(optionalProduct.get().getCode()).isEqualTo("P100"); + assertThat(optionalProduct.get().getPrice()) + .isEqualTo(new BigDecimal("14.50")); + }); + } +} +``` + +Here's what the test does: + +- `@SpringBootTest` starts the full Spring application context. +- The Testcontainers special JDBC URL (`jdbc:tc:mysql:8.0.32:///db`) in + `@TestPropertySource` spins up a MySQL container and configures it as the + datasource automatically. +- `@Testcontainers` and `@Container` manage the lifecycle of the Kafka + container. `@DynamicPropertySource` registers the Kafka bootstrap servers + with Spring so that the producer and consumer connect to the test container. +- `@BeforeEach` creates a `Product` record in the database before each test. +- The test sends a `ProductPriceChangedEvent` to the `product-price-changes` + topic using `KafkaTemplate`. Spring Boot converts the object to JSON using + `JsonSerializer`. +- Because Kafka message processing is asynchronous, the test uses + [Awaitility](http://www.awaitility.org/) to poll every 3 seconds (up to a + maximum of 10 seconds) until the product price in the database matches the + expected value. +- The property `spring.kafka.consumer.auto-offset-reset` is set to `earliest` + so that the listener consumes messages even if they're sent to the topic + before the listener is ready. This setting is helpful when running tests. + +## Run tests and next steps + +### Run the tests + +```console +$ ./mvnw test +``` + +Or with Gradle: + +```console +$ ./gradlew test +``` + +You should see the Kafka and MySQL Docker containers start and all tests pass. +After the tests finish, the containers stop and are removed automatically. + +### Summary + +Testing with real Kafka and MySQL instances gives you more confidence in the +correctness of your code than mocks or embedded alternatives. The +Testcontainers library manages the container lifecycle so that your integration +tests run against the same services you use in production. + +To learn more about Testcontainers, visit the +[Testcontainers overview](https://testcontainers.com/getting-started/). + +### Further reading + +- [Getting started with Testcontainers in a Java Spring Boot project](https://testcontainers.com/guides/testing-spring-boot-rest-api-using-testcontainers/) +- [The simplest way to replace H2 with a real database for testing](https://testcontainers.com/guides/replace-h2-with-real-database-for-testing/) +- [Awaitility](http://www.awaitility.org/) +- [Testcontainers Kafka module](https://java.testcontainers.org/modules/kafka/) +- [Testcontainers MySQL module](https://java.testcontainers.org/modules/databases/mysql/) diff --git a/content/guides/testcontainers-java-spring-boot-kafka/create-project.md b/content/guides/testcontainers-java-spring-boot-kafka/create-project.md deleted file mode 100644 index 52c0e1b7fb67..000000000000 --- a/content/guides/testcontainers-java-spring-boot-kafka/create-project.md +++ /dev/null @@ -1,296 +0,0 @@ ---- -title: Create the Spring Boot project -linkTitle: Create the project -description: Set up a Spring Boot project with Kafka, Spring Data JPA, and MySQL. -keywords: testcontainers, java, spring boot, kafka, mysql, project setup -weight: 10 ---- - -## Set up the project - -Create a Spring Boot project from [Spring Initializr](https://start.spring.io) -by selecting the **Spring for Apache Kafka**, **Spring Data JPA**, **MySQL Driver**, -and **Testcontainers** starters. - -Alternatively, clone the -[guide repository](https://github.com/testcontainers/tc-guide-testing-spring-boot-kafka-listener). - -After generating the application, add the Awaitility library as a test -dependency. You'll use it later to assert the expectations of an asynchronous -process flow. - -The key dependencies in `pom.xml` are: - -```xml - - 17 - 2.0.4 - - - - org.springframework.boot - spring-boot-starter-data-jpa - - - org.springframework.kafka - spring-kafka - - - com.mysql - mysql-connector-j - runtime - - - org.springframework.boot - spring-boot-starter-test - test - - - org.springframework.kafka - spring-kafka-test - test - - - org.testcontainers - testcontainers-junit-jupiter - test - - - org.testcontainers - testcontainers-kafka - test - - - org.testcontainers - testcontainers-mysql - test - - - org.awaitility - awaitility - test - - -``` - -Using the Testcontainers BOM (Bill of Materials) is recommended so that you -don't have to repeat the version for every Testcontainers module dependency. - -## Create the JPA entity - -The application listens to a topic called `product-price-changes`. When a -message arrives, it extracts the product code and price from the event payload -and updates the price for that product in the MySQL database. - -Create `Product.java`: - -```java -package com.testcontainers.demo; - -import jakarta.persistence.Column; -import jakarta.persistence.Entity; -import jakarta.persistence.GeneratedValue; -import jakarta.persistence.GenerationType; -import jakarta.persistence.Id; -import jakarta.persistence.Table; -import java.math.BigDecimal; - -@Entity -@Table(name = "products") -class Product { - - @Id - @GeneratedValue(strategy = GenerationType.IDENTITY) - private Long id; - - @Column(nullable = false, unique = true) - private String code; - - @Column(nullable = false) - private String name; - - @Column(nullable = false) - private BigDecimal price; - - public Product() {} - - public Product(Long id, String code, String name, BigDecimal price) { - this.id = id; - this.code = code; - this.name = name; - this.price = price; - } - - public Long getId() { - return id; - } - - public void setId(Long id) { - this.id = id; - } - - public String getCode() { - return code; - } - - public void setCode(String code) { - this.code = code; - } - - public String getName() { - return name; - } - - public void setName(String name) { - this.name = name; - } - - public BigDecimal getPrice() { - return price; - } - - public void setPrice(BigDecimal price) { - this.price = price; - } -} -``` - -## Create the Spring Data JPA repository - -Create a repository interface for the `Product` entity with a method to find a -product by code and a method to update the price for a given product code: - -```java -package com.testcontainers.demo; - -import java.math.BigDecimal; -import java.util.Optional; -import org.springframework.data.jpa.repository.JpaRepository; -import org.springframework.data.jpa.repository.Modifying; -import org.springframework.data.jpa.repository.Query; -import org.springframework.data.repository.query.Param; - -interface ProductRepository extends JpaRepository { - Optional findByCode(String code); - - @Modifying - @Query("update Product p set p.price = :price where p.code = :productCode") - void updateProductPrice( - @Param("productCode") String productCode, - @Param("price") BigDecimal price - ); -} -``` - -## Add a schema creation script - -Because the application doesn't use an in-memory database, you need to create -the MySQL tables. The recommended approach for production is a migration tool -like Flyway or Liquibase, but for this guide the built-in Spring Boot schema -initialization is sufficient. - -Create `src/main/resources/schema.sql`: - -```sql -create table products ( - id int NOT NULL AUTO_INCREMENT, - code varchar(255) not null, - name varchar(255) not null, - price numeric(5,2) not null, - PRIMARY KEY (id), - UNIQUE (code) -); -``` - -Enable schema initialization in `src/main/resources/application.properties`: - -```properties -spring.sql.init.mode=always -``` - -## Create the event payload - -Create a record named `ProductPriceChangedEvent` that represents the structure -of the event payload received from the Kafka topic: - -```java -package com.testcontainers.demo; - -import java.math.BigDecimal; - -record ProductPriceChangedEvent(String productCode, BigDecimal price) {} -``` - -The sender and receiver agree on the following JSON format: - -```json -{ - "productCode": "P100", - "price": 25.0 -} -``` - -## Implement the Kafka listener - -Create `ProductPriceChangedEventHandler.java`, which handles messages from the -`product-price-changes` topic and updates the product price in the database: - -```java -package com.testcontainers.demo; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.springframework.kafka.annotation.KafkaListener; -import org.springframework.stereotype.Component; -import org.springframework.transaction.annotation.Transactional; - -@Component -@Transactional -class ProductPriceChangedEventHandler { - - private static final Logger log = LoggerFactory.getLogger( - ProductPriceChangedEventHandler.class - ); - - private final ProductRepository productRepository; - - ProductPriceChangedEventHandler(ProductRepository productRepository) { - this.productRepository = productRepository; - } - - @KafkaListener(topics = "product-price-changes", groupId = "demo") - public void handle(ProductPriceChangedEvent event) { - log.info( - "Received a ProductPriceChangedEvent with productCode:{}: ", - event.productCode() - ); - productRepository.updateProductPrice(event.productCode(), event.price()); - } -} -``` - -The `@KafkaListener` annotation specifies the topic name to listen to. Spring -Kafka handles serialization and deserialization based on the properties -configured in `application.properties`. - -## Configure Kafka serialization - -Add the following Kafka properties to -`src/main/resources/application.properties`: - -```properties -######## Kafka Configuration ######### -spring.kafka.bootstrap-servers=localhost:9092 -spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer -spring.kafka.producer.value-serializer=org.springframework.kafka.support.serializer.JsonSerializer - -spring.kafka.consumer.group-id=demo -spring.kafka.consumer.auto-offset-reset=latest -spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer -spring.kafka.consumer.value-deserializer=org.springframework.kafka.support.serializer.JsonDeserializer -spring.kafka.consumer.properties.spring.json.trusted.packages=com.testcontainers.demo -``` - -The `productCode` key is (de)serialized using `StringSerializer`/`StringDeserializer`, -and the `ProductPriceChangedEvent` value is (de)serialized using -`JsonSerializer`/`JsonDeserializer`. diff --git a/content/guides/testcontainers-java-spring-boot-kafka/run-tests.md b/content/guides/testcontainers-java-spring-boot-kafka/run-tests.md deleted file mode 100644 index 6dd353b2dd02..000000000000 --- a/content/guides/testcontainers-java-spring-boot-kafka/run-tests.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -title: Run tests and next steps -linkTitle: Run tests -description: Run your Testcontainers-based Spring Boot Kafka integration tests and explore next steps. -keywords: testcontainers, java, spring boot, kafka, mysql, integration testing -weight: 30 ---- - -## Run the tests - -```console -$ ./mvnw test -``` - -Or with Gradle: - -```console -$ ./gradlew test -``` - -You should see the Kafka and MySQL Docker containers start and all tests pass. -After the tests finish, the containers stop and are removed automatically. - -## Summary - -Testing with real Kafka and MySQL instances gives you more confidence in the -correctness of your code than mocks or embedded alternatives. The -Testcontainers library manages the container lifecycle so that your integration -tests run against the same services you use in production. - -To learn more about Testcontainers, visit the -[Testcontainers overview](https://testcontainers.com/getting-started/). - -## Further reading - -- [Getting started with Testcontainers in a Java Spring Boot project](https://testcontainers.com/guides/testing-spring-boot-rest-api-using-testcontainers/) -- [The simplest way to replace H2 with a real database for testing](https://testcontainers.com/guides/replace-h2-with-real-database-for-testing/) -- [Awaitility](http://www.awaitility.org/) -- [Testcontainers Kafka module](https://java.testcontainers.org/modules/kafka/) -- [Testcontainers MySQL module](https://java.testcontainers.org/modules/databases/mysql/) diff --git a/content/guides/testcontainers-java-spring-boot-kafka/write-tests.md b/content/guides/testcontainers-java-spring-boot-kafka/write-tests.md deleted file mode 100644 index 2994367eec24..000000000000 --- a/content/guides/testcontainers-java-spring-boot-kafka/write-tests.md +++ /dev/null @@ -1,114 +0,0 @@ ---- -title: Write tests with Testcontainers -linkTitle: Write tests -description: Test the Spring Boot Kafka listener using Testcontainers Kafka and MySQL modules with Awaitility. -keywords: testcontainers, java, spring boot, kafka, mysql, awaitility, integration testing -weight: 20 ---- - -To test the Kafka listener, you need a running Kafka broker and a MySQL -database, plus a started Spring context. Testcontainers spins up both services -in Docker containers and `@DynamicPropertySource` connects them to Spring. - -## Write the test - -Create `ProductPriceChangedEventHandlerTest.java`: - -```java -package com.testcontainers.demo; - -import static java.util.concurrent.TimeUnit.SECONDS; -import static org.assertj.core.api.Assertions.assertThat; -import static org.awaitility.Awaitility.await; - -import java.math.BigDecimal; -import java.time.Duration; -import java.util.Optional; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.Test; -import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.kafka.core.KafkaTemplate; -import org.springframework.test.context.DynamicPropertyRegistry; -import org.springframework.test.context.DynamicPropertySource; -import org.springframework.test.context.TestPropertySource; -import org.testcontainers.kafka.ConfluentKafkaContainer; -import org.testcontainers.junit.jupiter.Container; -import org.testcontainers.junit.jupiter.Testcontainers; - -@SpringBootTest -@TestPropertySource( - properties = { - "spring.kafka.consumer.auto-offset-reset=earliest", - "spring.datasource.url=jdbc:tc:mysql:8.0.32:///db", - } -) -@Testcontainers -class ProductPriceChangedEventHandlerTest { - - @Container - static final ConfluentKafkaContainer kafka = - new ConfluentKafkaContainer("confluentinc/cp-kafka:7.8.0"); - - @DynamicPropertySource - static void overrideProperties(DynamicPropertyRegistry registry) { - registry.add("spring.kafka.bootstrap-servers", kafka::getBootstrapServers); - } - - @Autowired - private KafkaTemplate kafkaTemplate; - - @Autowired - private ProductRepository productRepository; - - @BeforeEach - void setUp() { - Product product = new Product(null, "P100", "Product One", BigDecimal.TEN); - productRepository.save(product); - } - - @Test - void shouldHandleProductPriceChangedEvent() { - ProductPriceChangedEvent event = new ProductPriceChangedEvent( - "P100", - new BigDecimal("14.50") - ); - - kafkaTemplate.send("product-price-changes", event.productCode(), event); - - await() - .pollInterval(Duration.ofSeconds(3)) - .atMost(10, SECONDS) - .untilAsserted(() -> { - Optional optionalProduct = productRepository.findByCode( - "P100" - ); - assertThat(optionalProduct).isPresent(); - assertThat(optionalProduct.get().getCode()).isEqualTo("P100"); - assertThat(optionalProduct.get().getPrice()) - .isEqualTo(new BigDecimal("14.50")); - }); - } -} -``` - -Here's what the test does: - -- `@SpringBootTest` starts the full Spring application context. -- The Testcontainers special JDBC URL (`jdbc:tc:mysql:8.0.32:///db`) in - `@TestPropertySource` spins up a MySQL container and configures it as the - datasource automatically. -- `@Testcontainers` and `@Container` manage the lifecycle of the Kafka - container. `@DynamicPropertySource` registers the Kafka bootstrap servers - with Spring so that the producer and consumer connect to the test container. -- `@BeforeEach` creates a `Product` record in the database before each test. -- The test sends a `ProductPriceChangedEvent` to the `product-price-changes` - topic using `KafkaTemplate`. Spring Boot converts the object to JSON using - `JsonSerializer`. -- Because Kafka message processing is asynchronous, the test uses - [Awaitility](http://www.awaitility.org/) to poll every 3 seconds (up to a - maximum of 10 seconds) until the product price in the database matches the - expected value. -- The property `spring.kafka.consumer.auto-offset-reset` is set to `earliest` - so that the listener consumes messages even if they're sent to the topic - before the listener is ready. This setting is helpful when running tests. diff --git a/content/guides/testcontainers-java-spring-boot-rest-api/_index.md b/content/guides/testcontainers-java-spring-boot-rest-api/_index.md index 23dfa0de0bb3..edc177db5bfd 100644 --- a/content/guides/testcontainers-java-spring-boot-rest-api/_index.md +++ b/content/guides/testcontainers-java-spring-boot-rest-api/_index.md @@ -6,14 +6,16 @@ keywords: testcontainers, java, spring boot, testing, postgresql, rest api, rest summary: | Learn how to create a Spring Boot REST API with Spring Data JPA and PostgreSQL, then test it using Testcontainers and REST Assured. -toc_min: 1 -toc_max: 2 -tags: [testing-with-docker] -languages: [java] +aliases: + - /guides/testcontainers-java-spring-boot-rest-api/create-project/ + - /guides/testcontainers-java-spring-boot-rest-api/run-tests/ + - /guides/testcontainers-java-spring-boot-rest-api/write-tests/ params: + tags: [testing] time: 25 minutes --- + In this guide, you will learn how to: @@ -32,3 +34,309 @@ In this guide, you will learn how to: > If you're new to Testcontainers, visit the > [Testcontainers overview](https://testcontainers.com/getting-started/) to learn more about > Testcontainers and the benefits of using it. + +## Create the Spring Boot project + +### Set up the project + +Create a Spring Boot project from [Spring Initializr](https://start.spring.io) +by selecting the **Spring Web**, **Spring Data JPA**, **PostgreSQL Driver**, and +**Testcontainers** starters. + +Alternatively, clone the +[guide repository](https://github.com/testcontainers/tc-guide-testing-spring-boot-rest-api). + +The key dependencies in `pom.xml` are: + +```xml + + 17 + 2.0.4 + + + + org.springframework.boot + spring-boot-starter-data-jpa + + + org.springframework.boot + spring-boot-starter-web + + + org.postgresql + postgresql + runtime + + + org.springframework.boot + spring-boot-starter-test + test + + + org.testcontainers + testcontainers-junit-jupiter + test + + + org.testcontainers + testcontainers-postgresql + test + + + io.rest-assured + rest-assured + test + + +``` + +Using the Testcontainers BOM (Bill of Materials) is recommended so that you +don't have to repeat the version for every Testcontainers module dependency. + +### Create the JPA entity + +Create `Customer.java`: + +```java +package com.testcontainers.demo; + +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.GeneratedValue; +import jakarta.persistence.GenerationType; +import jakarta.persistence.Id; +import jakarta.persistence.Table; + +@Entity +@Table(name = "customers") +class Customer { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(nullable = false) + private String name; + + @Column(nullable = false, unique = true) + private String email; + + public Customer() {} + + public Customer(Long id, String name, String email) { + this.id = id; + this.name = name; + this.email = email; + } + + public Long getId() { + return id; + } + + public void setId(Long id) { + this.id = id; + } + + public String getName() { + return name; + } + + public void setName(String name) { + this.name = name; + } + + public String getEmail() { + return email; + } + + public void setEmail(String email) { + this.email = email; + } +} +``` + +### Create the Spring Data JPA repository + +```java +package com.testcontainers.demo; + +import org.springframework.data.jpa.repository.JpaRepository; + +interface CustomerRepository extends JpaRepository {} +``` + +### Add the schema creation script + +Create `src/main/resources/schema.sql`: + +```sql +create table if not exists customers ( + id bigserial not null, + name varchar not null, + email varchar not null, + primary key (id), + UNIQUE (email) +); +``` + +Enable schema initialization in `src/main/resources/application.properties`: + +```properties +spring.sql.init.mode=always +``` + +### Create the REST API endpoint + +Create `CustomerController.java`: + +```java +package com.testcontainers.demo; + +import java.util.List; +import org.springframework.web.bind.annotation.GetMapping; +import org.springframework.web.bind.annotation.RestController; + +@RestController +class CustomerController { + + private final CustomerRepository repo; + + CustomerController(CustomerRepository repo) { + this.repo = repo; + } + + @GetMapping("/api/customers") + List getAll() { + return repo.findAll(); + } +} +``` + +## Write tests with Testcontainers + +To test the REST API, you need a running Postgres database and a started +Spring context. Testcontainers spins up Postgres in a Docker container and +`@DynamicPropertySource` connects it to Spring. + +### Write the test + +Create `CustomerControllerTest.java`: + +```java +package com.testcontainers.demo; + +import static io.restassured.RestAssured.given; +import static org.hamcrest.Matchers.hasSize; + +import io.restassured.RestAssured; +import io.restassured.http.ContentType; +import java.util.List; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.web.server.LocalServerPort; +import org.springframework.test.context.DynamicPropertyRegistry; +import org.springframework.test.context.DynamicPropertySource; +import org.testcontainers.postgresql.PostgreSQLContainer; + +@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT) +class CustomerControllerTest { + + @LocalServerPort + private Integer port; + + static PostgreSQLContainer postgres = new PostgreSQLContainer( + "postgres:16-alpine" + ); + + @BeforeAll + static void beforeAll() { + postgres.start(); + } + + @AfterAll + static void afterAll() { + postgres.stop(); + } + + @DynamicPropertySource + static void configureProperties(DynamicPropertyRegistry registry) { + registry.add("spring.datasource.url", postgres::getJdbcUrl); + registry.add("spring.datasource.username", postgres::getUsername); + registry.add("spring.datasource.password", postgres::getPassword); + } + + @Autowired + CustomerRepository customerRepository; + + @BeforeEach + void setUp() { + RestAssured.baseURI = "http://localhost:" + port; + customerRepository.deleteAll(); + } + + @Test + void shouldGetAllCustomers() { + List customers = List.of( + new Customer(null, "John", "john@mail.com"), + new Customer(null, "Dennis", "dennis@mail.com") + ); + customerRepository.saveAll(customers); + + given() + .contentType(ContentType.JSON) + .when() + .get("/api/customers") + .then() + .statusCode(200) + .body(".", hasSize(2)); + } +} +``` + +Here's what the test does: + +- `@SpringBootTest` starts the full application on a random port. +- A `PostgreSQLContainer` starts in `@BeforeAll` and stops in `@AfterAll`. +- `@DynamicPropertySource` registers the container's JDBC URL, username, and + password with Spring so that the datasource connects to the test container. +- `@BeforeEach` deletes all customer rows before each test to prevent test + pollution. +- `shouldGetAllCustomers()` inserts two customers, calls `GET /api/customers`, + and verifies the response contains 2 records. + +## Run tests and next steps + +### Run the tests + +```console +$ ./mvnw test +``` + +Or with Gradle: + +```console +$ ./gradlew test +``` + +You should see the Postgres Docker container start and all tests pass. After +the tests finish, the container stops and is removed automatically. + +### Summary + +The Testcontainers library helps you write integration tests by using the same +type of database (Postgres) that you use in production, instead of mocks or +in-memory databases. Because you test against real services, you're free to +refactor code and still verify that the application works as expected. + +To learn more about Testcontainers, visit the +[Testcontainers overview](https://testcontainers.com/getting-started/). + +### Further reading + +- [Testcontainers JUnit 5 quickstart](https://java.testcontainers.org/quickstart/junit_5_quickstart/) +- [Testcontainers Postgres module](https://java.testcontainers.org/modules/databases/postgres/) +- [Testcontainers JDBC support](https://java.testcontainers.org/modules/databases/jdbc/) diff --git a/content/guides/testcontainers-java-spring-boot-rest-api/create-project.md b/content/guides/testcontainers-java-spring-boot-rest-api/create-project.md deleted file mode 100644 index a700278a2145..000000000000 --- a/content/guides/testcontainers-java-spring-boot-rest-api/create-project.md +++ /dev/null @@ -1,182 +0,0 @@ ---- -title: Create the Spring Boot project -linkTitle: Create the project -description: Set up a Spring Boot project with Spring Data JPA, PostgreSQL, and a REST API. -keywords: testcontainers, java, spring boot, rest api, postgresql, project setup -weight: 10 ---- - -## Set up the project - -Create a Spring Boot project from [Spring Initializr](https://start.spring.io) -by selecting the **Spring Web**, **Spring Data JPA**, **PostgreSQL Driver**, and -**Testcontainers** starters. - -Alternatively, clone the -[guide repository](https://github.com/testcontainers/tc-guide-testing-spring-boot-rest-api). - -The key dependencies in `pom.xml` are: - -```xml - - 17 - 2.0.4 - - - - org.springframework.boot - spring-boot-starter-data-jpa - - - org.springframework.boot - spring-boot-starter-web - - - org.postgresql - postgresql - runtime - - - org.springframework.boot - spring-boot-starter-test - test - - - org.testcontainers - testcontainers-junit-jupiter - test - - - org.testcontainers - testcontainers-postgresql - test - - - io.rest-assured - rest-assured - test - - -``` - -Using the Testcontainers BOM (Bill of Materials) is recommended so that you -don't have to repeat the version for every Testcontainers module dependency. - -## Create the JPA entity - -Create `Customer.java`: - -```java -package com.testcontainers.demo; - -import jakarta.persistence.Column; -import jakarta.persistence.Entity; -import jakarta.persistence.GeneratedValue; -import jakarta.persistence.GenerationType; -import jakarta.persistence.Id; -import jakarta.persistence.Table; - -@Entity -@Table(name = "customers") -class Customer { - - @Id - @GeneratedValue(strategy = GenerationType.IDENTITY) - private Long id; - - @Column(nullable = false) - private String name; - - @Column(nullable = false, unique = true) - private String email; - - public Customer() {} - - public Customer(Long id, String name, String email) { - this.id = id; - this.name = name; - this.email = email; - } - - public Long getId() { - return id; - } - - public void setId(Long id) { - this.id = id; - } - - public String getName() { - return name; - } - - public void setName(String name) { - this.name = name; - } - - public String getEmail() { - return email; - } - - public void setEmail(String email) { - this.email = email; - } -} -``` - -## Create the Spring Data JPA repository - -```java -package com.testcontainers.demo; - -import org.springframework.data.jpa.repository.JpaRepository; - -interface CustomerRepository extends JpaRepository {} -``` - -## Add the schema creation script - -Create `src/main/resources/schema.sql`: - -```sql -create table if not exists customers ( - id bigserial not null, - name varchar not null, - email varchar not null, - primary key (id), - UNIQUE (email) -); -``` - -Enable schema initialization in `src/main/resources/application.properties`: - -```properties -spring.sql.init.mode=always -``` - -## Create the REST API endpoint - -Create `CustomerController.java`: - -```java -package com.testcontainers.demo; - -import java.util.List; -import org.springframework.web.bind.annotation.GetMapping; -import org.springframework.web.bind.annotation.RestController; - -@RestController -class CustomerController { - - private final CustomerRepository repo; - - CustomerController(CustomerRepository repo) { - this.repo = repo; - } - - @GetMapping("/api/customers") - List getAll() { - return repo.findAll(); - } -} -``` diff --git a/content/guides/testcontainers-java-spring-boot-rest-api/run-tests.md b/content/guides/testcontainers-java-spring-boot-rest-api/run-tests.md deleted file mode 100644 index c8c7ba2acbf8..000000000000 --- a/content/guides/testcontainers-java-spring-boot-rest-api/run-tests.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -title: Run tests and next steps -linkTitle: Run tests -description: Run your Testcontainers-based Spring Boot integration tests and explore next steps. -keywords: testcontainers, java, spring boot, rest api, postgresql, integration testing -weight: 30 ---- - -## Run the tests - -```console -$ ./mvnw test -``` - -Or with Gradle: - -```console -$ ./gradlew test -``` - -You should see the Postgres Docker container start and all tests pass. After -the tests finish, the container stops and is removed automatically. - -## Summary - -The Testcontainers library helps you write integration tests by using the same -type of database (Postgres) that you use in production, instead of mocks or -in-memory databases. Because you test against real services, you're free to -refactor code and still verify that the application works as expected. - -To learn more about Testcontainers, visit the -[Testcontainers overview](https://testcontainers.com/getting-started/). - -## Further reading - -- [Testcontainers JUnit 5 quickstart](https://java.testcontainers.org/quickstart/junit_5_quickstart/) -- [Testcontainers Postgres module](https://java.testcontainers.org/modules/databases/postgres/) -- [Testcontainers JDBC support](https://java.testcontainers.org/modules/databases/jdbc/) diff --git a/content/guides/testcontainers-java-spring-boot-rest-api/write-tests.md b/content/guides/testcontainers-java-spring-boot-rest-api/write-tests.md deleted file mode 100644 index 2f5f15b1f204..000000000000 --- a/content/guides/testcontainers-java-spring-boot-rest-api/write-tests.md +++ /dev/null @@ -1,101 +0,0 @@ ---- -title: Write tests with Testcontainers -linkTitle: Write tests -description: Test the Spring Boot REST API using Testcontainers and REST Assured. -keywords: testcontainers, java, spring boot, rest api, rest assured, integration testing -weight: 20 ---- - -To test the REST API, you need a running Postgres database and a started -Spring context. Testcontainers spins up Postgres in a Docker container and -`@DynamicPropertySource` connects it to Spring. - -## Write the test - -Create `CustomerControllerTest.java`: - -```java -package com.testcontainers.demo; - -import static io.restassured.RestAssured.given; -import static org.hamcrest.Matchers.hasSize; - -import io.restassured.RestAssured; -import io.restassured.http.ContentType; -import java.util.List; -import org.junit.jupiter.api.AfterAll; -import org.junit.jupiter.api.BeforeAll; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.Test; -import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.test.web.server.LocalServerPort; -import org.springframework.test.context.DynamicPropertyRegistry; -import org.springframework.test.context.DynamicPropertySource; -import org.testcontainers.postgresql.PostgreSQLContainer; - -@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT) -class CustomerControllerTest { - - @LocalServerPort - private Integer port; - - static PostgreSQLContainer postgres = new PostgreSQLContainer( - "postgres:16-alpine" - ); - - @BeforeAll - static void beforeAll() { - postgres.start(); - } - - @AfterAll - static void afterAll() { - postgres.stop(); - } - - @DynamicPropertySource - static void configureProperties(DynamicPropertyRegistry registry) { - registry.add("spring.datasource.url", postgres::getJdbcUrl); - registry.add("spring.datasource.username", postgres::getUsername); - registry.add("spring.datasource.password", postgres::getPassword); - } - - @Autowired - CustomerRepository customerRepository; - - @BeforeEach - void setUp() { - RestAssured.baseURI = "http://localhost:" + port; - customerRepository.deleteAll(); - } - - @Test - void shouldGetAllCustomers() { - List customers = List.of( - new Customer(null, "John", "john@mail.com"), - new Customer(null, "Dennis", "dennis@mail.com") - ); - customerRepository.saveAll(customers); - - given() - .contentType(ContentType.JSON) - .when() - .get("/api/customers") - .then() - .statusCode(200) - .body(".", hasSize(2)); - } -} -``` - -Here's what the test does: - -- `@SpringBootTest` starts the full application on a random port. -- A `PostgreSQLContainer` starts in `@BeforeAll` and stops in `@AfterAll`. -- `@DynamicPropertySource` registers the container's JDBC URL, username, and - password with Spring so that the datasource connects to the test container. -- `@BeforeEach` deletes all customer rows before each test to prevent test - pollution. -- `shouldGetAllCustomers()` inserts two customers, calls `GET /api/customers`, - and verifies the response contains 2 records. diff --git a/content/guides/testcontainers-java-wiremock/_index.md b/content/guides/testcontainers-java-wiremock/_index.md index 5de620502c56..536683ddc493 100644 --- a/content/guides/testcontainers-java-wiremock/_index.md +++ b/content/guides/testcontainers-java-wiremock/_index.md @@ -7,14 +7,16 @@ summary: | Learn how to create a Spring Boot application that integrates with external REST APIs, then test those integrations using Testcontainers and WireMock. -toc_min: 1 -toc_max: 2 -tags: [testing-with-docker] -languages: [java] +aliases: + - /guides/testcontainers-java-wiremock/create-project/ + - /guides/testcontainers-java-wiremock/run-tests/ + - /guides/testcontainers-java-wiremock/write-tests/ params: + tags: [testing] time: 20 minutes --- + In this guide, you'll learn how to: @@ -33,3 +35,736 @@ In this guide, you'll learn how to: > If you're new to Testcontainers, visit the > [Testcontainers overview](https://testcontainers.com/getting-started/) to learn more about > Testcontainers and the benefits of using it. + +## Create the Spring Boot project + +### Set up the project + +Create a Spring Boot project from [Spring Initializr](https://start.spring.io) +by selecting the **Spring Web** and **Testcontainers** starters. + +Alternatively, clone the +[guide repository](https://github.com/testcontainers/tc-guide-testing-rest-api-integrations-using-wiremock). + +After generating the project, add the **REST Assured**, **WireMock**, and +**WireMock Testcontainers module** libraries as test dependencies. The key +dependencies in `pom.xml` are: + +```xml + + 17 + 2.0.4 + 1.0-alpha-13 + + + + org.springframework.boot + spring-boot-starter-web + + + org.springframework.boot + spring-boot-starter-test + test + + + org.testcontainers + testcontainers-junit-jupiter + test + + + org.wiremock + wiremock-standalone + 3.6.0 + test + + + org.wiremock.integrations.testcontainers + wiremock-testcontainers-module + ${wiremock-testcontainers.version} + test + + + io.rest-assured + rest-assured + test + + +``` + +Using the Testcontainers BOM (Bill of Materials) is recommended so that you +don't have to repeat the version for every Testcontainers module dependency. + +This guide builds an application that manages video albums. A third-party REST +API handles photo assets. For demonstration purposes, the application uses the +publicly available [JSONPlaceholder](https://jsonplaceholder.typicode.com/) API +as a photo service. + +The application exposes a `GET /api/albums/{albumId}` endpoint that calls the +photo service to fetch photos for a given album. +[WireMock](https://wiremock.org/) is a tool for building mock APIs. +Testcontainers provides a +[WireMock module](https://testcontainers.com/modules/wiremock/) that runs +WireMock as a Docker container. + +### Create the Album and Photo models + +Create `Album.java` using Java records: + +```java +package com.testcontainers.demo; + +import java.util.List; + +public record Album(Long albumId, List photos) {} + +record Photo(Long id, String title, String url, String thumbnailUrl) {} +``` + +### Create the PhotoServiceClient + +Create `PhotoServiceClient.java`, which uses `RestTemplate` to fetch photos for +a given album ID: + +```java +package com.testcontainers.demo; + +import java.util.List; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.boot.web.client.RestTemplateBuilder; +import org.springframework.core.ParameterizedTypeReference; +import org.springframework.http.HttpMethod; +import org.springframework.http.ResponseEntity; +import org.springframework.stereotype.Service; +import org.springframework.web.client.RestTemplate; + +@Service +class PhotoServiceClient { + + private final String baseUrl; + private final RestTemplate restTemplate; + + PhotoServiceClient( + @Value("${photos.api.base-url}") String baseUrl, + RestTemplateBuilder builder + ) { + this.baseUrl = baseUrl; + this.restTemplate = builder.build(); + } + + List getPhotos(Long albumId) { + String url = baseUrl + "/albums/{albumId}/photos"; + ResponseEntity> response = restTemplate.exchange( + url, + HttpMethod.GET, + null, + new ParameterizedTypeReference<>() {}, + albumId + ); + return response.getBody(); + } +} +``` + +The photo service base URL is externalized as a configuration property. Add the +following entry to `src/main/resources/application.properties`: + +```properties +photos.api.base-url=https://jsonplaceholder.typicode.com +``` + +### Create the REST API endpoint + +Create `AlbumController.java`: + +```java +package com.testcontainers.demo; + +import java.util.List; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.http.ResponseEntity; +import org.springframework.web.bind.annotation.GetMapping; +import org.springframework.web.bind.annotation.PathVariable; +import org.springframework.web.bind.annotation.RequestMapping; +import org.springframework.web.bind.annotation.RestController; +import org.springframework.web.client.RestClientResponseException; + +@RestController +@RequestMapping("/api") +class AlbumController { + + private static final Logger logger = LoggerFactory.getLogger( + AlbumController.class + ); + + private final PhotoServiceClient photoServiceClient; + + AlbumController(PhotoServiceClient photoServiceClient) { + this.photoServiceClient = photoServiceClient; + } + + @GetMapping("/albums/{albumId}") + public ResponseEntity getAlbumById(@PathVariable Long albumId) { + try { + List photos = photoServiceClient.getPhotos(albumId); + return ResponseEntity.ok(new Album(albumId, photos)); + } catch (RestClientResponseException e) { + logger.error("Failed to get photos", e); + return new ResponseEntity<>(e.getStatusCode()); + } + } +} +``` + +This endpoint calls the photo service for a given album ID and returns a +response like: + +```json +{ + "albumId": 1, + "photos": [ + { + "id": 51, + "title": "non sunt voluptatem placeat consequuntur rem incidunt", + "url": "https://via.placeholder.com/600/8e973b", + "thumbnailUrl": "https://via.placeholder.com/150/8e973b" + }, + { + "id": 52, + "title": "eveniet pariatur quia nobis reiciendis laboriosam ea", + "url": "https://via.placeholder.com/600/121fa4", + "thumbnailUrl": "https://via.placeholder.com/150/121fa4" + } + ] +} +``` + +## Write tests with WireMock and Testcontainers + +Mocking external API interactions at the HTTP protocol level, rather than +mocking Java methods, lets you verify marshalling and unmarshalling behavior and +simulate network issues. + +### Test using WireMock JUnit 5 extension + +WireMock provides a JUnit 5 extension that starts an in-process WireMock server. +You can configure stub responses using the WireMock Java API. + +Create `AlbumControllerTest.java`: + +```java +package com.testcontainers.demo; + +import static com.github.tomakehurst.wiremock.client.WireMock.aResponse; +import static com.github.tomakehurst.wiremock.client.WireMock.urlMatching; +import static com.github.tomakehurst.wiremock.core.WireMockConfiguration.wireMockConfig; +import static io.restassured.RestAssured.given; +import static org.hamcrest.CoreMatchers.is; +import static org.hamcrest.Matchers.hasSize; +import static org.springframework.boot.test.context.SpringBootTest.WebEnvironment.RANDOM_PORT; + +import com.github.tomakehurst.wiremock.client.WireMock; +import com.github.tomakehurst.wiremock.junit5.WireMockExtension; +import io.restassured.RestAssured; +import io.restassured.http.ContentType; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.RegisterExtension; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.web.server.LocalServerPort; +import org.springframework.http.MediaType; +import org.springframework.test.context.DynamicPropertyRegistry; +import org.springframework.test.context.DynamicPropertySource; + +@SpringBootTest(webEnvironment = RANDOM_PORT) +class AlbumControllerTest { + + @LocalServerPort + private Integer port; + + @RegisterExtension + static WireMockExtension wireMock = WireMockExtension + .newInstance() + .options(wireMockConfig().dynamicPort()) + .build(); + + @DynamicPropertySource + static void configureProperties(DynamicPropertyRegistry registry) { + registry.add("photos.api.base-url", wireMock::baseUrl); + } + + @BeforeEach + void setUp() { + RestAssured.port = port; + } + + @Test + void shouldGetAlbumById() { + Long albumId = 1L; + + wireMock.stubFor( + WireMock + .get(urlMatching("/albums/" + albumId + "/photos")) + .willReturn( + aResponse() + .withHeader("Content-Type", MediaType.APPLICATION_JSON_VALUE) + .withBody( + """ + [ + { + "id": 1, + "title": "accusamus beatae ad facilis cum similique qui sunt", + "url": "https://via.placeholder.com/600/92c952", + "thumbnailUrl": "https://via.placeholder.com/150/92c952" + }, + { + "id": 2, + "title": "reprehenderit est deserunt velit ipsam", + "url": "https://via.placeholder.com/600/771796", + "thumbnailUrl": "https://via.placeholder.com/150/771796" + } + ] + """ + ) + ) + ); + + given() + .contentType(ContentType.JSON) + .when() + .get("/api/albums/{albumId}", albumId) + .then() + .statusCode(200) + .body("albumId", is(albumId.intValue())) + .body("photos", hasSize(2)); + } + + @Test + void shouldReturnServerErrorWhenPhotoServiceCallFailed() { + Long albumId = 2L; + wireMock.stubFor( + WireMock + .get(urlMatching("/albums/" + albumId + "/photos")) + .willReturn(aResponse().withStatus(500)) + ); + + given() + .contentType(ContentType.JSON) + .when() + .get("/api/albums/{albumId}", albumId) + .then() + .statusCode(500); + } +} +``` + +Here's what the test does: + +- `@SpringBootTest` starts the full application on a random port. +- `@RegisterExtension` creates a `WireMockExtension` that starts WireMock on a + dynamic port. +- `@DynamicPropertySource` overrides `photos.api.base-url` to point at the + WireMock endpoint, so the application talks to WireMock instead of the real + photo service. +- `shouldGetAlbumById()` configures a stub response for + `/albums/{albumId}/photos`, sends a request to the application's + `/api/albums/{albumId}` endpoint, and verifies the response body. +- `shouldReturnServerErrorWhenPhotoServiceCallFailed()` configures WireMock to + return a 500 status and verifies that the application propagates that status to + the caller. + +### Stub using JSON mapping files + +Instead of using the WireMock Java API, you can configure stubs with JSON +mapping files. Create +`src/test/resources/wiremock/mappings/get-album-photos.json`: + +```json +{ + "mappings": [ + { + "request": { + "method": "GET", + "urlPattern": "/albums/([0-9]+)/photos" + }, + "response": { + "status": 200, + "headers": { + "Content-Type": "application/json" + }, + "bodyFileName": "album-photos-resp-200.json" + } + }, + { + "request": { + "method": "GET", + "urlPattern": "/albums/2/photos" + }, + "response": { + "status": 500, + "headers": { + "Content-Type": "application/json" + } + } + }, + { + "request": { + "method": "GET", + "urlPattern": "/albums/3/photos" + }, + "response": { + "status": 200, + "headers": { + "Content-Type": "application/json" + }, + "jsonBody": [] + } + } + ] +} +``` + +Create the response body file at +`src/test/resources/wiremock/__files/album-photos-resp-200.json`: + +```json +[ + { + "id": 1, + "title": "accusamus beatae ad facilis cum similique qui sunt", + "url": "https://via.placeholder.com/600/92c952", + "thumbnailUrl": "https://via.placeholder.com/150/92c952" + }, + { + "id": 2, + "title": "reprehenderit est deserunt velit ipsam", + "url": "https://via.placeholder.com/600/771796", + "thumbnailUrl": "https://via.placeholder.com/150/771796" + } +] +``` + +Initialize WireMock to load stubs from the mapping files: + +```java +@RegisterExtension +static WireMockExtension wireMockServer = WireMockExtension + .newInstance() + .options( + wireMockConfig().dynamicPort().usingFilesUnderClasspath("wiremock") + ) + .build(); +``` + +With mapping-based stubs in place, create +`AlbumControllerWireMockMappingTests.java`: + +```java +package com.testcontainers.demo; + +import static com.github.tomakehurst.wiremock.core.WireMockConfiguration.wireMockConfig; +import static io.restassured.RestAssured.given; +import static org.hamcrest.CoreMatchers.is; +import static org.hamcrest.Matchers.hasSize; +import static org.springframework.boot.test.context.SpringBootTest.WebEnvironment.RANDOM_PORT; + +import com.github.tomakehurst.wiremock.junit5.WireMockExtension; +import io.restassured.RestAssured; +import io.restassured.http.ContentType; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.RegisterExtension; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.web.server.LocalServerPort; +import org.springframework.test.context.DynamicPropertyRegistry; +import org.springframework.test.context.DynamicPropertySource; + +@SpringBootTest(webEnvironment = RANDOM_PORT) +class AlbumControllerWireMockMappingTests { + + @LocalServerPort + private Integer port; + + @RegisterExtension + static WireMockExtension wireMockServer = WireMockExtension + .newInstance() + .options( + wireMockConfig().dynamicPort().usingFilesUnderClasspath("wiremock") + ) + .build(); + + @DynamicPropertySource + static void configureProperties(DynamicPropertyRegistry registry) { + registry.add("photos.api.base-url", wireMockServer::baseUrl); + } + + @BeforeEach + void setUp() { + RestAssured.port = port; + } + + @Test + void shouldGetAlbumById() { + Long albumId = 1L; + + given() + .contentType(ContentType.JSON) + .when() + .get("/api/albums/{albumId}", albumId) + .then() + .statusCode(200) + .body("albumId", is(albumId.intValue())) + .body("photos", hasSize(2)); + } + + @Test + void shouldReturnServerErrorWhenPhotoServiceCallFailed() { + Long albumId = 2L; + + given() + .contentType(ContentType.JSON) + .when() + .get("/api/albums/{albumId}", albumId) + .then() + .statusCode(500); + } + + @Test + void shouldReturnEmptyPhotos() { + Long albumId = 3L; + + given() + .contentType(ContentType.JSON) + .when() + .get("/api/albums/{albumId}", albumId) + .then() + .statusCode(200) + .body("albumId", is(albumId.intValue())) + .body("photos", hasSize(0)); + } +} +``` + +The tests don't need inline stub definitions because WireMock loads the mappings +automatically from the classpath. + +### Test using the Testcontainers WireMock module + +The [Testcontainers WireMock module](https://testcontainers.com/modules/wiremock/) +provisions WireMock as a standalone Docker container, based on +[WireMock Docker](https://github.com/wiremock/wiremock-docker). This approach is +useful when you want complete isolation between the test JVM and the mock server. + +Create a mock configuration file at +`src/test/resources/com/testcontainers/demo/AlbumControllerTestcontainersTests/mocks-config.json`: + +```json +{ + "mappings": [ + { + "request": { + "method": "GET", + "urlPattern": "/albums/([0-9]+)/photos" + }, + "response": { + "status": 200, + "headers": { + "Content-Type": "application/json" + }, + "bodyFileName": "album-photos-response.json" + } + }, + { + "request": { + "method": "GET", + "urlPattern": "/albums/2/photos" + }, + "response": { + "status": 500, + "headers": { + "Content-Type": "application/json" + } + } + }, + { + "request": { + "method": "GET", + "urlPattern": "/albums/3/photos" + }, + "response": { + "status": 200, + "headers": { + "Content-Type": "application/json" + }, + "jsonBody": [] + } + } + ] +} +``` + +Create the response body file at +`src/test/resources/com/testcontainers/demo/AlbumControllerTestcontainersTests/album-photos-response.json`: + +```json +[ + { + "id": 1, + "title": "accusamus beatae ad facilis cum similique qui sunt", + "url": "https://via.placeholder.com/600/92c952", + "thumbnailUrl": "https://via.placeholder.com/150/92c952" + }, + { + "id": 2, + "title": "reprehenderit est deserunt velit ipsam", + "url": "https://via.placeholder.com/600/771796", + "thumbnailUrl": "https://via.placeholder.com/150/771796" + } +] +``` + +Create `AlbumControllerTestcontainersTests.java`: + +```java +package com.testcontainers.demo; + +import static io.restassured.RestAssured.given; +import static org.hamcrest.CoreMatchers.is; +import static org.hamcrest.Matchers.hasSize; +import static org.springframework.boot.test.context.SpringBootTest.WebEnvironment.RANDOM_PORT; + +import io.restassured.RestAssured; +import io.restassured.http.ContentType; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.web.server.LocalServerPort; +import org.springframework.test.context.DynamicPropertyRegistry; +import org.springframework.test.context.DynamicPropertySource; +import org.testcontainers.junit.jupiter.Container; +import org.testcontainers.junit.jupiter.Testcontainers; +import org.wiremock.integrations.testcontainers.WireMockContainer; + +@SpringBootTest(webEnvironment = RANDOM_PORT) +@Testcontainers +class AlbumControllerTestcontainersTests { + + @LocalServerPort + private Integer port; + + @Container + static WireMockContainer wiremockServer = new WireMockContainer( + "wiremock/wiremock:3.6.0" + ) + .withMapping( + "photos-by-album", + AlbumControllerTestcontainersTests.class, + "mocks-config.json" + ) + .withFileFromResource( + "album-photos-response.json", + AlbumControllerTestcontainersTests.class, + "album-photos-response.json" + ); + + @DynamicPropertySource + static void configureProperties(DynamicPropertyRegistry registry) { + registry.add("photos.api.base-url", wiremockServer::getBaseUrl); + } + + @BeforeEach + void setUp() { + RestAssured.port = port; + } + + @Test + void shouldGetAlbumById() { + Long albumId = 1L; + + given() + .contentType(ContentType.JSON) + .when() + .get("/api/albums/{albumId}", albumId) + .then() + .statusCode(200) + .body("albumId", is(albumId.intValue())) + .body("photos", hasSize(2)); + } + + @Test + void shouldReturnServerErrorWhenPhotoServiceCallFailed() { + Long albumId = 2L; + + given() + .contentType(ContentType.JSON) + .when() + .get("/api/albums/{albumId}", albumId) + .then() + .statusCode(500); + } + + @Test + void shouldReturnEmptyPhotos() { + Long albumId = 3L; + + given() + .contentType(ContentType.JSON) + .when() + .get("/api/albums/{albumId}", albumId) + .then() + .statusCode(200) + .body("albumId", is(albumId.intValue())) + .body("photos", hasSize(0)); + } +} +``` + +Here's what the test does: + +- The `@Testcontainers` and `@Container` annotations start a + `WireMockContainer` using the `wiremock/wiremock:3.6.0` Docker image. +- `withMapping()` loads stub mappings from `mocks-config.json`, and + `withFileFromResource()` loads the response body file. +- `@DynamicPropertySource` overrides `photos.api.base-url` to point at the + WireMock container's base URL. +- The tests don't contain inline stub definitions because WireMock loads them + from the JSON configuration files. + +## Run tests and next steps + +### Run the tests + +```console +$ ./mvnw test +``` + +Or with Gradle: + +```console +$ ./gradlew test +``` + +You should see the WireMock Docker container start in the console output. It +acts as the photo service, serving mock responses based on the configured +expectations. All tests should pass. + +### Summary + +You built a Spring Boot application that integrates with an external REST API, +then tested that integration using three different approaches: + +- WireMock JUnit 5 extension with inline stubs +- WireMock JUnit 5 extension with JSON mapping files +- Testcontainers WireMock module running WireMock in a Docker container + +Testing at the HTTP protocol level instead of mocking Java methods lets you +catch serialization issues and simulate realistic failure scenarios. + +To learn more about Testcontainers, visit the +[Testcontainers overview](https://testcontainers.com/getting-started/). + +### Further reading + +- [Testcontainers WireMock module](https://testcontainers.com/modules/wiremock/) +- [WireMock documentation](https://wiremock.org/docs/) +- [Testcontainers JUnit 5 quickstart](https://java.testcontainers.org/quickstart/junit_5_quickstart/) diff --git a/content/guides/testcontainers-java-wiremock/create-project.md b/content/guides/testcontainers-java-wiremock/create-project.md deleted file mode 100644 index da34c798577f..000000000000 --- a/content/guides/testcontainers-java-wiremock/create-project.md +++ /dev/null @@ -1,208 +0,0 @@ ---- -title: Create the Spring Boot project -linkTitle: Create the project -description: Set up a Spring Boot project with an external REST API integration using WireMock and Testcontainers. -keywords: testcontainers, java, spring boot, wiremock, rest api, project setup -weight: 10 ---- - -## Set up the project - -Create a Spring Boot project from [Spring Initializr](https://start.spring.io) -by selecting the **Spring Web** and **Testcontainers** starters. - -Alternatively, clone the -[guide repository](https://github.com/testcontainers/tc-guide-testing-rest-api-integrations-using-wiremock). - -After generating the project, add the **REST Assured**, **WireMock**, and -**WireMock Testcontainers module** libraries as test dependencies. The key -dependencies in `pom.xml` are: - -```xml - - 17 - 2.0.4 - 1.0-alpha-13 - - - - org.springframework.boot - spring-boot-starter-web - - - org.springframework.boot - spring-boot-starter-test - test - - - org.testcontainers - testcontainers-junit-jupiter - test - - - org.wiremock - wiremock-standalone - 3.6.0 - test - - - org.wiremock.integrations.testcontainers - wiremock-testcontainers-module - ${wiremock-testcontainers.version} - test - - - io.rest-assured - rest-assured - test - - -``` - -Using the Testcontainers BOM (Bill of Materials) is recommended so that you -don't have to repeat the version for every Testcontainers module dependency. - -This guide builds an application that manages video albums. A third-party REST -API handles photo assets. For demonstration purposes, the application uses the -publicly available [JSONPlaceholder](https://jsonplaceholder.typicode.com/) API -as a photo service. - -The application exposes a `GET /api/albums/{albumId}` endpoint that calls the -photo service to fetch photos for a given album. -[WireMock](https://wiremock.org/) is a tool for building mock APIs. -Testcontainers provides a -[WireMock module](https://testcontainers.com/modules/wiremock/) that runs -WireMock as a Docker container. - -## Create the Album and Photo models - -Create `Album.java` using Java records: - -```java -package com.testcontainers.demo; - -import java.util.List; - -public record Album(Long albumId, List photos) {} - -record Photo(Long id, String title, String url, String thumbnailUrl) {} -``` - -## Create the PhotoServiceClient - -Create `PhotoServiceClient.java`, which uses `RestTemplate` to fetch photos for -a given album ID: - -```java -package com.testcontainers.demo; - -import java.util.List; -import org.springframework.beans.factory.annotation.Value; -import org.springframework.boot.web.client.RestTemplateBuilder; -import org.springframework.core.ParameterizedTypeReference; -import org.springframework.http.HttpMethod; -import org.springframework.http.ResponseEntity; -import org.springframework.stereotype.Service; -import org.springframework.web.client.RestTemplate; - -@Service -class PhotoServiceClient { - - private final String baseUrl; - private final RestTemplate restTemplate; - - PhotoServiceClient( - @Value("${photos.api.base-url}") String baseUrl, - RestTemplateBuilder builder - ) { - this.baseUrl = baseUrl; - this.restTemplate = builder.build(); - } - - List getPhotos(Long albumId) { - String url = baseUrl + "/albums/{albumId}/photos"; - ResponseEntity> response = restTemplate.exchange( - url, - HttpMethod.GET, - null, - new ParameterizedTypeReference<>() {}, - albumId - ); - return response.getBody(); - } -} -``` - -The photo service base URL is externalized as a configuration property. Add the -following entry to `src/main/resources/application.properties`: - -```properties -photos.api.base-url=https://jsonplaceholder.typicode.com -``` - -## Create the REST API endpoint - -Create `AlbumController.java`: - -```java -package com.testcontainers.demo; - -import java.util.List; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.springframework.http.ResponseEntity; -import org.springframework.web.bind.annotation.GetMapping; -import org.springframework.web.bind.annotation.PathVariable; -import org.springframework.web.bind.annotation.RequestMapping; -import org.springframework.web.bind.annotation.RestController; -import org.springframework.web.client.RestClientResponseException; - -@RestController -@RequestMapping("/api") -class AlbumController { - - private static final Logger logger = LoggerFactory.getLogger( - AlbumController.class - ); - - private final PhotoServiceClient photoServiceClient; - - AlbumController(PhotoServiceClient photoServiceClient) { - this.photoServiceClient = photoServiceClient; - } - - @GetMapping("/albums/{albumId}") - public ResponseEntity getAlbumById(@PathVariable Long albumId) { - try { - List photos = photoServiceClient.getPhotos(albumId); - return ResponseEntity.ok(new Album(albumId, photos)); - } catch (RestClientResponseException e) { - logger.error("Failed to get photos", e); - return new ResponseEntity<>(e.getStatusCode()); - } - } -} -``` - -This endpoint calls the photo service for a given album ID and returns a -response like: - -```json -{ - "albumId": 1, - "photos": [ - { - "id": 51, - "title": "non sunt voluptatem placeat consequuntur rem incidunt", - "url": "https://via.placeholder.com/600/8e973b", - "thumbnailUrl": "https://via.placeholder.com/150/8e973b" - }, - { - "id": 52, - "title": "eveniet pariatur quia nobis reiciendis laboriosam ea", - "url": "https://via.placeholder.com/600/121fa4", - "thumbnailUrl": "https://via.placeholder.com/150/121fa4" - } - ] -} -``` diff --git a/content/guides/testcontainers-java-wiremock/run-tests.md b/content/guides/testcontainers-java-wiremock/run-tests.md deleted file mode 100644 index ea147efbf284..000000000000 --- a/content/guides/testcontainers-java-wiremock/run-tests.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -title: Run tests and next steps -linkTitle: Run tests -description: Run your Testcontainers WireMock integration tests and explore next steps. -keywords: testcontainers, java, spring boot, wiremock, integration testing -weight: 30 ---- - -## Run the tests - -```console -$ ./mvnw test -``` - -Or with Gradle: - -```console -$ ./gradlew test -``` - -You should see the WireMock Docker container start in the console output. It -acts as the photo service, serving mock responses based on the configured -expectations. All tests should pass. - -## Summary - -You built a Spring Boot application that integrates with an external REST API, -then tested that integration using three different approaches: - -- WireMock JUnit 5 extension with inline stubs -- WireMock JUnit 5 extension with JSON mapping files -- Testcontainers WireMock module running WireMock in a Docker container - -Testing at the HTTP protocol level instead of mocking Java methods lets you -catch serialization issues and simulate realistic failure scenarios. - -To learn more about Testcontainers, visit the -[Testcontainers overview](https://testcontainers.com/getting-started/). - -## Further reading - -- [Testcontainers WireMock module](https://testcontainers.com/modules/wiremock/) -- [WireMock documentation](https://wiremock.org/docs/) -- [Testcontainers JUnit 5 quickstart](https://java.testcontainers.org/quickstart/junit_5_quickstart/) diff --git a/content/guides/testcontainers-java-wiremock/write-tests.md b/content/guides/testcontainers-java-wiremock/write-tests.md deleted file mode 100644 index c8d0bd915dfa..000000000000 --- a/content/guides/testcontainers-java-wiremock/write-tests.md +++ /dev/null @@ -1,496 +0,0 @@ ---- -title: Write tests with WireMock and Testcontainers -linkTitle: Write tests -description: Test external REST API integrations using WireMock with both the JUnit 5 extension and the Testcontainers WireMock module. -keywords: testcontainers, java, spring boot, wiremock, junit, rest api, integration testing -weight: 20 ---- - -Mocking external API interactions at the HTTP protocol level, rather than -mocking Java methods, lets you verify marshalling and unmarshalling behavior and -simulate network issues. - -## Test using WireMock JUnit 5 extension - -WireMock provides a JUnit 5 extension that starts an in-process WireMock server. -You can configure stub responses using the WireMock Java API. - -Create `AlbumControllerTest.java`: - -```java -package com.testcontainers.demo; - -import static com.github.tomakehurst.wiremock.client.WireMock.aResponse; -import static com.github.tomakehurst.wiremock.client.WireMock.urlMatching; -import static com.github.tomakehurst.wiremock.core.WireMockConfiguration.wireMockConfig; -import static io.restassured.RestAssured.given; -import static org.hamcrest.CoreMatchers.is; -import static org.hamcrest.Matchers.hasSize; -import static org.springframework.boot.test.context.SpringBootTest.WebEnvironment.RANDOM_PORT; - -import com.github.tomakehurst.wiremock.client.WireMock; -import com.github.tomakehurst.wiremock.junit5.WireMockExtension; -import io.restassured.RestAssured; -import io.restassured.http.ContentType; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.extension.RegisterExtension; -import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.test.web.server.LocalServerPort; -import org.springframework.http.MediaType; -import org.springframework.test.context.DynamicPropertyRegistry; -import org.springframework.test.context.DynamicPropertySource; - -@SpringBootTest(webEnvironment = RANDOM_PORT) -class AlbumControllerTest { - - @LocalServerPort - private Integer port; - - @RegisterExtension - static WireMockExtension wireMock = WireMockExtension - .newInstance() - .options(wireMockConfig().dynamicPort()) - .build(); - - @DynamicPropertySource - static void configureProperties(DynamicPropertyRegistry registry) { - registry.add("photos.api.base-url", wireMock::baseUrl); - } - - @BeforeEach - void setUp() { - RestAssured.port = port; - } - - @Test - void shouldGetAlbumById() { - Long albumId = 1L; - - wireMock.stubFor( - WireMock - .get(urlMatching("/albums/" + albumId + "/photos")) - .willReturn( - aResponse() - .withHeader("Content-Type", MediaType.APPLICATION_JSON_VALUE) - .withBody( - """ - [ - { - "id": 1, - "title": "accusamus beatae ad facilis cum similique qui sunt", - "url": "https://via.placeholder.com/600/92c952", - "thumbnailUrl": "https://via.placeholder.com/150/92c952" - }, - { - "id": 2, - "title": "reprehenderit est deserunt velit ipsam", - "url": "https://via.placeholder.com/600/771796", - "thumbnailUrl": "https://via.placeholder.com/150/771796" - } - ] - """ - ) - ) - ); - - given() - .contentType(ContentType.JSON) - .when() - .get("/api/albums/{albumId}", albumId) - .then() - .statusCode(200) - .body("albumId", is(albumId.intValue())) - .body("photos", hasSize(2)); - } - - @Test - void shouldReturnServerErrorWhenPhotoServiceCallFailed() { - Long albumId = 2L; - wireMock.stubFor( - WireMock - .get(urlMatching("/albums/" + albumId + "/photos")) - .willReturn(aResponse().withStatus(500)) - ); - - given() - .contentType(ContentType.JSON) - .when() - .get("/api/albums/{albumId}", albumId) - .then() - .statusCode(500); - } -} -``` - -Here's what the test does: - -- `@SpringBootTest` starts the full application on a random port. -- `@RegisterExtension` creates a `WireMockExtension` that starts WireMock on a - dynamic port. -- `@DynamicPropertySource` overrides `photos.api.base-url` to point at the - WireMock endpoint, so the application talks to WireMock instead of the real - photo service. -- `shouldGetAlbumById()` configures a stub response for - `/albums/{albumId}/photos`, sends a request to the application's - `/api/albums/{albumId}` endpoint, and verifies the response body. -- `shouldReturnServerErrorWhenPhotoServiceCallFailed()` configures WireMock to - return a 500 status and verifies that the application propagates that status to - the caller. - -## Stub using JSON mapping files - -Instead of using the WireMock Java API, you can configure stubs with JSON -mapping files. Create -`src/test/resources/wiremock/mappings/get-album-photos.json`: - -```json -{ - "mappings": [ - { - "request": { - "method": "GET", - "urlPattern": "/albums/([0-9]+)/photos" - }, - "response": { - "status": 200, - "headers": { - "Content-Type": "application/json" - }, - "bodyFileName": "album-photos-resp-200.json" - } - }, - { - "request": { - "method": "GET", - "urlPattern": "/albums/2/photos" - }, - "response": { - "status": 500, - "headers": { - "Content-Type": "application/json" - } - } - }, - { - "request": { - "method": "GET", - "urlPattern": "/albums/3/photos" - }, - "response": { - "status": 200, - "headers": { - "Content-Type": "application/json" - }, - "jsonBody": [] - } - } - ] -} -``` - -Create the response body file at -`src/test/resources/wiremock/__files/album-photos-resp-200.json`: - -```json -[ - { - "id": 1, - "title": "accusamus beatae ad facilis cum similique qui sunt", - "url": "https://via.placeholder.com/600/92c952", - "thumbnailUrl": "https://via.placeholder.com/150/92c952" - }, - { - "id": 2, - "title": "reprehenderit est deserunt velit ipsam", - "url": "https://via.placeholder.com/600/771796", - "thumbnailUrl": "https://via.placeholder.com/150/771796" - } -] -``` - -Initialize WireMock to load stubs from the mapping files: - -```java -@RegisterExtension -static WireMockExtension wireMockServer = WireMockExtension - .newInstance() - .options( - wireMockConfig().dynamicPort().usingFilesUnderClasspath("wiremock") - ) - .build(); -``` - -With mapping-based stubs in place, create -`AlbumControllerWireMockMappingTests.java`: - -```java -package com.testcontainers.demo; - -import static com.github.tomakehurst.wiremock.core.WireMockConfiguration.wireMockConfig; -import static io.restassured.RestAssured.given; -import static org.hamcrest.CoreMatchers.is; -import static org.hamcrest.Matchers.hasSize; -import static org.springframework.boot.test.context.SpringBootTest.WebEnvironment.RANDOM_PORT; - -import com.github.tomakehurst.wiremock.junit5.WireMockExtension; -import io.restassured.RestAssured; -import io.restassured.http.ContentType; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.extension.RegisterExtension; -import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.test.web.server.LocalServerPort; -import org.springframework.test.context.DynamicPropertyRegistry; -import org.springframework.test.context.DynamicPropertySource; - -@SpringBootTest(webEnvironment = RANDOM_PORT) -class AlbumControllerWireMockMappingTests { - - @LocalServerPort - private Integer port; - - @RegisterExtension - static WireMockExtension wireMockServer = WireMockExtension - .newInstance() - .options( - wireMockConfig().dynamicPort().usingFilesUnderClasspath("wiremock") - ) - .build(); - - @DynamicPropertySource - static void configureProperties(DynamicPropertyRegistry registry) { - registry.add("photos.api.base-url", wireMockServer::baseUrl); - } - - @BeforeEach - void setUp() { - RestAssured.port = port; - } - - @Test - void shouldGetAlbumById() { - Long albumId = 1L; - - given() - .contentType(ContentType.JSON) - .when() - .get("/api/albums/{albumId}", albumId) - .then() - .statusCode(200) - .body("albumId", is(albumId.intValue())) - .body("photos", hasSize(2)); - } - - @Test - void shouldReturnServerErrorWhenPhotoServiceCallFailed() { - Long albumId = 2L; - - given() - .contentType(ContentType.JSON) - .when() - .get("/api/albums/{albumId}", albumId) - .then() - .statusCode(500); - } - - @Test - void shouldReturnEmptyPhotos() { - Long albumId = 3L; - - given() - .contentType(ContentType.JSON) - .when() - .get("/api/albums/{albumId}", albumId) - .then() - .statusCode(200) - .body("albumId", is(albumId.intValue())) - .body("photos", hasSize(0)); - } -} -``` - -The tests don't need inline stub definitions because WireMock loads the mappings -automatically from the classpath. - -## Test using the Testcontainers WireMock module - -The [Testcontainers WireMock module](https://testcontainers.com/modules/wiremock/) -provisions WireMock as a standalone Docker container, based on -[WireMock Docker](https://github.com/wiremock/wiremock-docker). This approach is -useful when you want complete isolation between the test JVM and the mock server. - -Create a mock configuration file at -`src/test/resources/com/testcontainers/demo/AlbumControllerTestcontainersTests/mocks-config.json`: - -```json -{ - "mappings": [ - { - "request": { - "method": "GET", - "urlPattern": "/albums/([0-9]+)/photos" - }, - "response": { - "status": 200, - "headers": { - "Content-Type": "application/json" - }, - "bodyFileName": "album-photos-response.json" - } - }, - { - "request": { - "method": "GET", - "urlPattern": "/albums/2/photos" - }, - "response": { - "status": 500, - "headers": { - "Content-Type": "application/json" - } - } - }, - { - "request": { - "method": "GET", - "urlPattern": "/albums/3/photos" - }, - "response": { - "status": 200, - "headers": { - "Content-Type": "application/json" - }, - "jsonBody": [] - } - } - ] -} -``` - -Create the response body file at -`src/test/resources/com/testcontainers/demo/AlbumControllerTestcontainersTests/album-photos-response.json`: - -```json -[ - { - "id": 1, - "title": "accusamus beatae ad facilis cum similique qui sunt", - "url": "https://via.placeholder.com/600/92c952", - "thumbnailUrl": "https://via.placeholder.com/150/92c952" - }, - { - "id": 2, - "title": "reprehenderit est deserunt velit ipsam", - "url": "https://via.placeholder.com/600/771796", - "thumbnailUrl": "https://via.placeholder.com/150/771796" - } -] -``` - -Create `AlbumControllerTestcontainersTests.java`: - -```java -package com.testcontainers.demo; - -import static io.restassured.RestAssured.given; -import static org.hamcrest.CoreMatchers.is; -import static org.hamcrest.Matchers.hasSize; -import static org.springframework.boot.test.context.SpringBootTest.WebEnvironment.RANDOM_PORT; - -import io.restassured.RestAssured; -import io.restassured.http.ContentType; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.Test; -import org.springframework.boot.test.context.SpringBootTest; -import org.springframework.boot.test.web.server.LocalServerPort; -import org.springframework.test.context.DynamicPropertyRegistry; -import org.springframework.test.context.DynamicPropertySource; -import org.testcontainers.junit.jupiter.Container; -import org.testcontainers.junit.jupiter.Testcontainers; -import org.wiremock.integrations.testcontainers.WireMockContainer; - -@SpringBootTest(webEnvironment = RANDOM_PORT) -@Testcontainers -class AlbumControllerTestcontainersTests { - - @LocalServerPort - private Integer port; - - @Container - static WireMockContainer wiremockServer = new WireMockContainer( - "wiremock/wiremock:3.6.0" - ) - .withMapping( - "photos-by-album", - AlbumControllerTestcontainersTests.class, - "mocks-config.json" - ) - .withFileFromResource( - "album-photos-response.json", - AlbumControllerTestcontainersTests.class, - "album-photos-response.json" - ); - - @DynamicPropertySource - static void configureProperties(DynamicPropertyRegistry registry) { - registry.add("photos.api.base-url", wiremockServer::getBaseUrl); - } - - @BeforeEach - void setUp() { - RestAssured.port = port; - } - - @Test - void shouldGetAlbumById() { - Long albumId = 1L; - - given() - .contentType(ContentType.JSON) - .when() - .get("/api/albums/{albumId}", albumId) - .then() - .statusCode(200) - .body("albumId", is(albumId.intValue())) - .body("photos", hasSize(2)); - } - - @Test - void shouldReturnServerErrorWhenPhotoServiceCallFailed() { - Long albumId = 2L; - - given() - .contentType(ContentType.JSON) - .when() - .get("/api/albums/{albumId}", albumId) - .then() - .statusCode(500); - } - - @Test - void shouldReturnEmptyPhotos() { - Long albumId = 3L; - - given() - .contentType(ContentType.JSON) - .when() - .get("/api/albums/{albumId}", albumId) - .then() - .statusCode(200) - .body("albumId", is(albumId.intValue())) - .body("photos", hasSize(0)); - } -} -``` - -Here's what the test does: - -- The `@Testcontainers` and `@Container` annotations start a - `WireMockContainer` using the `wiremock/wiremock:3.6.0` Docker image. -- `withMapping()` loads stub mappings from `mocks-config.json`, and - `withFileFromResource()` loads the response body file. -- `@DynamicPropertySource` overrides `photos.api.base-url` to point at the - WireMock container's base URL. -- The tests don't contain inline stub definitions because WireMock loads them - from the JSON configuration files. diff --git a/content/guides/testcontainers-nodejs-getting-started/_index.md b/content/guides/testcontainers-nodejs-getting-started/_index.md index 891fa3266e59..f59c191ae6cb 100644 --- a/content/guides/testcontainers-nodejs-getting-started/_index.md +++ b/content/guides/testcontainers-nodejs-getting-started/_index.md @@ -6,14 +6,16 @@ keywords: testcontainers, nodejs, javascript, testing, postgresql, integration t summary: | Learn how to create a Node.js application and test database interactions using Testcontainers for Node.js with a real PostgreSQL instance. -toc_min: 1 -toc_max: 2 -tags: [testing-with-docker] -languages: [js] +aliases: + - /guides/testcontainers-nodejs-getting-started/create-project/ + - /guides/testcontainers-nodejs-getting-started/run-tests/ + - /guides/testcontainers-nodejs-getting-started/write-tests/ params: + tags: [testing] time: 15 minutes --- + In this guide, you will learn how to: @@ -32,3 +34,166 @@ In this guide, you will learn how to: > If you're new to Testcontainers, visit the > [Testcontainers overview](https://testcontainers.com/getting-started/) to learn more about > Testcontainers and the benefits of using it. + +## Create the Node.js project + +### Initialize the project + +Create a new Node.js project: + +```console +$ npm init -y +``` + +Add `pg`, `jest`, and `@testcontainers/postgresql` as dependencies: + +```console +$ npm install pg --save +$ npm install jest @testcontainers/postgresql --save-dev +``` + +### Implement the customer repository + +Create `src/customer-repository.js` with functions to manage customers in +PostgreSQL: + +```javascript +async function createCustomerTable(client) { + const sql = + "CREATE TABLE IF NOT EXISTS customers (id INT NOT NULL, name VARCHAR NOT NULL, PRIMARY KEY (id))"; + await client.query(sql); +} + +async function createCustomer(client, customer) { + const sql = "INSERT INTO customers (id, name) VALUES($1, $2)"; + await client.query(sql, [customer.id, customer.name]); +} + +async function getCustomers(client) { + const sql = "SELECT * FROM customers"; + const result = await client.query(sql); + return result.rows; +} + +module.exports = { createCustomerTable, createCustomer, getCustomers }; +``` + +The module provides three functions: + +- `createCustomerTable()` creates the `customers` table if it doesn't exist. +- `createCustomer()` inserts a customer record. +- `getCustomers()` fetches all customer records. + +## Write tests with Testcontainers + +Create `src/customer-repository.test.js` with the test: + +```javascript +const { Client } = require("pg"); +const { PostgreSqlContainer } = require("@testcontainers/postgresql"); +const { + createCustomerTable, + createCustomer, + getCustomers, +} = require("./customer-repository"); + +describe("Customer Repository", () => { + jest.setTimeout(60000); + + let postgresContainer; + let postgresClient; + + beforeAll(async () => { + postgresContainer = await new PostgreSqlContainer().start(); + postgresClient = new Client({ + connectionString: postgresContainer.getConnectionUri(), + }); + await postgresClient.connect(); + await createCustomerTable(postgresClient); + }); + + afterAll(async () => { + await postgresClient.end(); + await postgresContainer.stop(); + }); + + it("should create and return multiple customers", async () => { + const customer1 = { id: 1, name: "John Doe" }; + const customer2 = { id: 2, name: "Jane Doe" }; + + await createCustomer(postgresClient, customer1); + await createCustomer(postgresClient, customer2); + + const customers = await getCustomers(postgresClient); + expect(customers).toEqual([customer1, customer2]); + }); +}); +``` + +Here's what the test does: + +- The `beforeAll` block starts a real PostgreSQL container using + `PostgreSqlContainer`. It then creates a `pg` client connected to the + container and sets up the `customers` table. +- The `afterAll` block closes the client connection and stops the container. +- The test inserts two customers, fetches all customers, and asserts the + results match. + +The test timeout is set to 60 seconds to allow time for the container to start +on the first run (when the Docker image needs to be pulled). + +## Run tests and next steps + +### Run the tests + +Add the test script to `package.json` if it isn't there already: + +```json +{ + "scripts": { + "test": "jest" + } +} +``` + +Then run the tests: + +```console +$ npm test +``` + +You should see output like: + +```text + PASS src/customer-repository.test.js + Customer Repository + ✓ should create and return multiple customers (5 ms) + +Test Suites: 1 passed, 1 total +Tests: 1 passed, 1 total +``` + +To see what Testcontainers is doing under the hood — which containers it +starts, what versions it uses — set the `DEBUG` environment variable: + +```console +$ DEBUG=testcontainers* npm test +``` + +### Summary + +The Testcontainers for Node.js library helps you write integration tests using +the same type of database (Postgres) that you use in production, instead of +mocks. Because you aren't using mocks and instead talk to real services, you're +free to refactor code and still verify that the application works as expected. + +In addition to PostgreSQL, Testcontainers provides dedicated +[modules](https://github.com/testcontainers/testcontainers-node/tree/main/packages/modules) +for many SQL databases, NoSQL databases, messaging queues, and more. + +To learn more about Testcontainers, visit the +[Testcontainers overview](https://testcontainers.com/getting-started/). + +### Further reading + +- [Testcontainers for Node.js documentation](https://node.testcontainers.org) diff --git a/content/guides/testcontainers-nodejs-getting-started/create-project.md b/content/guides/testcontainers-nodejs-getting-started/create-project.md deleted file mode 100644 index e6a6e39f448c..000000000000 --- a/content/guides/testcontainers-nodejs-getting-started/create-project.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -title: Create the Node.js project -linkTitle: Create the project -description: Set up a Node.js project with a PostgreSQL-backed customer repository. -keywords: testcontainers, node.js, nodejs, postgresql, getting started, project setup -weight: 10 ---- - -## Initialize the project - -Create a new Node.js project: - -```console -$ npm init -y -``` - -Add `pg`, `jest`, and `@testcontainers/postgresql` as dependencies: - -```console -$ npm install pg --save -$ npm install jest @testcontainers/postgresql --save-dev -``` - -## Implement the customer repository - -Create `src/customer-repository.js` with functions to manage customers in -PostgreSQL: - -```javascript -async function createCustomerTable(client) { - const sql = - "CREATE TABLE IF NOT EXISTS customers (id INT NOT NULL, name VARCHAR NOT NULL, PRIMARY KEY (id))"; - await client.query(sql); -} - -async function createCustomer(client, customer) { - const sql = "INSERT INTO customers (id, name) VALUES($1, $2)"; - await client.query(sql, [customer.id, customer.name]); -} - -async function getCustomers(client) { - const sql = "SELECT * FROM customers"; - const result = await client.query(sql); - return result.rows; -} - -module.exports = { createCustomerTable, createCustomer, getCustomers }; -``` - -The module provides three functions: - -- `createCustomerTable()` creates the `customers` table if it doesn't exist. -- `createCustomer()` inserts a customer record. -- `getCustomers()` fetches all customer records. diff --git a/content/guides/testcontainers-nodejs-getting-started/run-tests.md b/content/guides/testcontainers-nodejs-getting-started/run-tests.md deleted file mode 100644 index 798dd24a3090..000000000000 --- a/content/guides/testcontainers-nodejs-getting-started/run-tests.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -title: Run tests and next steps -linkTitle: Run tests -description: Run your Testcontainers-based integration tests and explore next steps. -keywords: testcontainers, node.js, nodejs, postgresql, integration testing, run tests -weight: 30 ---- - -## Run the tests - -Add the test script to `package.json` if it isn't there already: - -```json -{ - "scripts": { - "test": "jest" - } -} -``` - -Then run the tests: - -```console -$ npm test -``` - -You should see output like: - -```text - PASS src/customer-repository.test.js - Customer Repository - ✓ should create and return multiple customers (5 ms) - -Test Suites: 1 passed, 1 total -Tests: 1 passed, 1 total -``` - -To see what Testcontainers is doing under the hood — which containers it -starts, what versions it uses — set the `DEBUG` environment variable: - -```console -$ DEBUG=testcontainers* npm test -``` - -## Summary - -The Testcontainers for Node.js library helps you write integration tests using -the same type of database (Postgres) that you use in production, instead of -mocks. Because you aren't using mocks and instead talk to real services, you're -free to refactor code and still verify that the application works as expected. - -In addition to PostgreSQL, Testcontainers provides dedicated -[modules](https://github.com/testcontainers/testcontainers-node/tree/main/packages/modules) -for many SQL databases, NoSQL databases, messaging queues, and more. - -To learn more about Testcontainers, visit the -[Testcontainers overview](https://testcontainers.com/getting-started/). - -## Further reading - -- [Testcontainers for Node.js documentation](https://node.testcontainers.org) diff --git a/content/guides/testcontainers-nodejs-getting-started/write-tests.md b/content/guides/testcontainers-nodejs-getting-started/write-tests.md deleted file mode 100644 index 61f591f232de..000000000000 --- a/content/guides/testcontainers-nodejs-getting-started/write-tests.md +++ /dev/null @@ -1,63 +0,0 @@ ---- -title: Write tests with Testcontainers -linkTitle: Write tests -description: Write integration tests using Testcontainers for Node.js and Jest with a real PostgreSQL database. -keywords: testcontainers, node.js, nodejs, postgresql, jest, integration testing -weight: 20 ---- - -Create `src/customer-repository.test.js` with the test: - -```javascript -const { Client } = require("pg"); -const { PostgreSqlContainer } = require("@testcontainers/postgresql"); -const { - createCustomerTable, - createCustomer, - getCustomers, -} = require("./customer-repository"); - -describe("Customer Repository", () => { - jest.setTimeout(60000); - - let postgresContainer; - let postgresClient; - - beforeAll(async () => { - postgresContainer = await new PostgreSqlContainer().start(); - postgresClient = new Client({ - connectionString: postgresContainer.getConnectionUri(), - }); - await postgresClient.connect(); - await createCustomerTable(postgresClient); - }); - - afterAll(async () => { - await postgresClient.end(); - await postgresContainer.stop(); - }); - - it("should create and return multiple customers", async () => { - const customer1 = { id: 1, name: "John Doe" }; - const customer2 = { id: 2, name: "Jane Doe" }; - - await createCustomer(postgresClient, customer1); - await createCustomer(postgresClient, customer2); - - const customers = await getCustomers(postgresClient); - expect(customers).toEqual([customer1, customer2]); - }); -}); -``` - -Here's what the test does: - -- The `beforeAll` block starts a real PostgreSQL container using - `PostgreSqlContainer`. It then creates a `pg` client connected to the - container and sets up the `customers` table. -- The `afterAll` block closes the client connection and stops the container. -- The test inserts two customers, fetches all customers, and asserts the - results match. - -The test timeout is set to 60 seconds to allow time for the container to start -on the first run (when the Docker image needs to be pulled). diff --git a/content/guides/testcontainers-python-getting-started/_index.md b/content/guides/testcontainers-python-getting-started/_index.md index 6a00efa89fcc..a2cf612e56e9 100644 --- a/content/guides/testcontainers-python-getting-started/_index.md +++ b/content/guides/testcontainers-python-getting-started/_index.md @@ -6,14 +6,16 @@ keywords: testcontainers, python, testing, postgresql, integration testing, pyte summary: | Learn how to create a Python application and test database interactions using Testcontainers for Python with a real PostgreSQL instance. -toc_min: 1 -toc_max: 2 -tags: [testing-with-docker] -languages: [python] +aliases: + - /guides/testcontainers-python-getting-started/create-project/ + - /guides/testcontainers-python-getting-started/run-tests/ + - /guides/testcontainers-python-getting-started/write-tests/ params: + tags: [testing] time: 15 minutes --- + In this guide, you will learn how to: @@ -33,3 +35,276 @@ In this guide, you will learn how to: > If you're new to Testcontainers, visit the > [Testcontainers overview](https://testcontainers.com/getting-started/) to learn more about > Testcontainers and the benefits of using it. + +## Create the Python project + +### Initialize the project + +Start by creating a Python project with a virtual environment: + +```console +$ mkdir tc-python-demo +$ cd tc-python-demo +$ python3 -m venv venv +$ source venv/bin/activate +``` + +This guide uses [psycopg3](https://www.psycopg.org/psycopg3/) to interact +with the Postgres database, [pytest](https://pytest.org/) for testing, and +[testcontainers-python](https://testcontainers-python.readthedocs.io/) for +running a PostgreSQL database in a container. + +Install the dependencies: + +```console +$ pip install "psycopg[binary]" pytest testcontainers[postgres] +$ pip freeze > requirements.txt +``` + +The `pip freeze` command generates a `requirements.txt` file so that others +can install the same package versions using `pip install -r requirements.txt`. + +### Create the database helper + +Create a `db/connection.py` file with a function to get a database connection: + +```python +import os + +import psycopg + + +def get_connection(): + host = os.getenv("DB_HOST", "localhost") + port = os.getenv("DB_PORT", "5432") + username = os.getenv("DB_USERNAME", "postgres") + password = os.getenv("DB_PASSWORD", "postgres") + database = os.getenv("DB_NAME", "postgres") + return psycopg.connect(f"host={host} dbname={database} user={username} password={password} port={port}") +``` + +Instead of hard-coding the database connection parameters, the function uses +environment variables. This makes it possible to run the application in +different environments without changing code. + +### Create the business logic + +Create a `customers/customers.py` file and define the `Customer` class: + +```python +class Customer: + def __init__(self, cust_id, name, email): + self.id = cust_id + self.name = name + self.email = email + + def __str__(self): + return f"Customer({self.id}, {self.name}, {self.email})" +``` + +Add a `create_table()` function to create the `customers` table: + +```python +from db.connection import get_connection + + +def create_table(): + with get_connection() as conn: + with conn.cursor() as cur: + cur.execute(""" + CREATE TABLE customers ( + id serial PRIMARY KEY, + name varchar not null, + email varchar not null unique) + """) + conn.commit() +``` + +The function obtains a database connection using `get_connection()` and creates +the `customers` table. The `with` statement automatically closes the connection +when done. + +Add the remaining CRUD functions: + +```python +def create_customer(name, email): + with get_connection() as conn: + with conn.cursor() as cur: + cur.execute( + "INSERT INTO customers (name, email) VALUES (%s, %s)", (name, email)) + conn.commit() + + +def get_all_customers() -> list[Customer]: + with get_connection() as conn: + with conn.cursor() as cur: + cur.execute("SELECT * FROM customers") + return [Customer(cid, name, email) for cid, name, email in cur] + + +def get_customer_by_email(email) -> Customer: + with get_connection() as conn: + with conn.cursor() as cur: + cur.execute("SELECT id, name, email FROM customers WHERE email = %s", (email,)) + (cid, name, email) = cur.fetchone() + return Customer(cid, name, email) + + +def delete_all_customers(): + with get_connection() as conn: + with conn.cursor() as cur: + cur.execute("DELETE FROM customers") + conn.commit() +``` + +> [!NOTE] +> To keep it straightforward for this guide, each function creates a new +> connection. In a real-world application, use a connection pool to reuse +> connections. + +## Write tests with Testcontainers + +You'll create a PostgreSQL container using Testcontainers and use it for all +the tests. Before each test, you'll delete all customer records so that tests +run with a clean database. + +### Set up pytest fixtures + +This guide uses [pytest fixtures](https://pytest.org/en/stable/how-to/fixtures.html) +for setup and teardown logic. A recommended approach is to use +[finalizers](https://pytest.org/en/stable/how-to/fixtures.html#adding-finalizers-directly) +to guarantee cleanup runs even if setup fails: + +```python +@pytest.fixture +def setup(request): + # setup code + + def cleanup(): + # teardown code + + request.addfinalizer(cleanup) + return some_value +``` + +### Create the test file + +Create a `tests/__init__.py` file with empty content to enable pytest +[auto-discovery](https://pytest.org/explanation/goodpractices.html#test-discovery). + +Then create `tests/test_customers.py` with the fixtures: + +```python +import os +import pytest +from testcontainers.postgres import PostgresContainer + +from customers import customers + +postgres = PostgresContainer("postgres:16-alpine") + + +@pytest.fixture(scope="module", autouse=True) +def setup(request): + postgres.start() + + def remove_container(): + postgres.stop() + + request.addfinalizer(remove_container) + os.environ["DB_CONN"] = postgres.get_connection_url() + os.environ["DB_HOST"] = postgres.get_container_host_ip() + os.environ["DB_PORT"] = str(postgres.get_exposed_port(5432)) + os.environ["DB_USERNAME"] = postgres.username + os.environ["DB_PASSWORD"] = postgres.password + os.environ["DB_NAME"] = postgres.dbname + customers.create_table() + + +@pytest.fixture(scope="function", autouse=True) +def setup_data(): + customers.delete_all_customers() +``` + +Here's what the fixtures do: + +- The `setup` fixture has `scope="module"`, so it runs once for all tests in + the file. It starts a PostgreSQL container, sets environment variables with + the connection details, and creates the `customers` table. A cleanup + function removes the container after all tests complete. +- The `setup_data` fixture has `scope="function"`, so it runs before every + test. It deletes all records to give each test a clean database. + +### Write the tests + +Add the test functions to the same file: + +```python +def test_get_all_customers(): + customers.create_customer("Siva", "siva@gmail.com") + customers.create_customer("James", "james@gmail.com") + customers_list = customers.get_all_customers() + assert len(customers_list) == 2 + + +def test_get_customer_by_email(): + customers.create_customer("John", "john@gmail.com") + customer = customers.get_customer_by_email("john@gmail.com") + assert customer.name == "John" + assert customer.email == "john@gmail.com" +``` + +- `test_get_all_customers()` inserts two customer records, fetches all + customers, and asserts the count. +- `test_get_customer_by_email()` inserts a customer, fetches it by email, and + asserts the details. + +Because `setup_data` deletes all records before each test, the tests can run in +any order. + +## Run tests and next steps + +### Run the tests + +Run the tests using pytest: + +```console +$ pytest -v +``` + +You should see output similar to: + +```text +============================= test session starts ============================== +platform linux -- Python 3.13.x, pytest-9.x.x +collected 2 items + +tests/test_customers.py::test_get_all_customers PASSED [ 50%] +tests/test_customers.py::test_get_customer_by_email PASSED [100%] + +============================== 2 passed in 1.90s =============================== +``` + +The tests run against a real PostgreSQL database instead of mocks, which gives +more confidence in the implementation. + +### Summary + +The Testcontainers for Python library helps you write integration tests using the +same type of database (Postgres) that you use in production, instead of mocks. +Because you aren't using mocks and instead talk to real services, you're free +to refactor code and still verify that the application works as expected. + +In addition to PostgreSQL, Testcontainers for Python provides modules for many +SQL databases, NoSQL databases, messaging queues, and more. You can use +Testcontainers to run any containerized dependency for your tests. + +To learn more about Testcontainers, visit the +[Testcontainers overview](https://testcontainers.com/getting-started/). + +### Further reading + +- [testcontainers-python documentation](https://testcontainers-python.readthedocs.io/) +- [Getting started with Testcontainers for Go](/guides/testcontainers-go-getting-started/) +- [Getting started with Testcontainers for Java](https://testcontainers.com/guides/getting-started-with-testcontainers-for-java/) +- [Getting started with Testcontainers for Node.js](https://testcontainers.com/guides/getting-started-with-testcontainers-for-nodejs/) diff --git a/content/guides/testcontainers-python-getting-started/create-project.md b/content/guides/testcontainers-python-getting-started/create-project.md deleted file mode 100644 index 022cfad7016e..000000000000 --- a/content/guides/testcontainers-python-getting-started/create-project.md +++ /dev/null @@ -1,131 +0,0 @@ ---- -title: Create the Python project -linkTitle: Create the project -description: Set up a Python project with a PostgreSQL-backed customer service. -keywords: testcontainers, python, postgresql, getting started, project setup -weight: 10 ---- - -## Initialize the project - -Start by creating a Python project with a virtual environment: - -```console -$ mkdir tc-python-demo -$ cd tc-python-demo -$ python3 -m venv venv -$ source venv/bin/activate -``` - -This guide uses [psycopg3](https://www.psycopg.org/psycopg3/) to interact -with the Postgres database, [pytest](https://pytest.org/) for testing, and -[testcontainers-python](https://testcontainers-python.readthedocs.io/) for -running a PostgreSQL database in a container. - -Install the dependencies: - -```console -$ pip install "psycopg[binary]" pytest testcontainers[postgres] -$ pip freeze > requirements.txt -``` - -The `pip freeze` command generates a `requirements.txt` file so that others -can install the same package versions using `pip install -r requirements.txt`. - -## Create the database helper - -Create a `db/connection.py` file with a function to get a database connection: - -```python -import os - -import psycopg - - -def get_connection(): - host = os.getenv("DB_HOST", "localhost") - port = os.getenv("DB_PORT", "5432") - username = os.getenv("DB_USERNAME", "postgres") - password = os.getenv("DB_PASSWORD", "postgres") - database = os.getenv("DB_NAME", "postgres") - return psycopg.connect(f"host={host} dbname={database} user={username} password={password} port={port}") -``` - -Instead of hard-coding the database connection parameters, the function uses -environment variables. This makes it possible to run the application in -different environments without changing code. - -## Create the business logic - -Create a `customers/customers.py` file and define the `Customer` class: - -```python -class Customer: - def __init__(self, cust_id, name, email): - self.id = cust_id - self.name = name - self.email = email - - def __str__(self): - return f"Customer({self.id}, {self.name}, {self.email})" -``` - -Add a `create_table()` function to create the `customers` table: - -```python -from db.connection import get_connection - - -def create_table(): - with get_connection() as conn: - with conn.cursor() as cur: - cur.execute(""" - CREATE TABLE customers ( - id serial PRIMARY KEY, - name varchar not null, - email varchar not null unique) - """) - conn.commit() -``` - -The function obtains a database connection using `get_connection()` and creates -the `customers` table. The `with` statement automatically closes the connection -when done. - -Add the remaining CRUD functions: - -```python -def create_customer(name, email): - with get_connection() as conn: - with conn.cursor() as cur: - cur.execute( - "INSERT INTO customers (name, email) VALUES (%s, %s)", (name, email)) - conn.commit() - - -def get_all_customers() -> list[Customer]: - with get_connection() as conn: - with conn.cursor() as cur: - cur.execute("SELECT * FROM customers") - return [Customer(cid, name, email) for cid, name, email in cur] - - -def get_customer_by_email(email) -> Customer: - with get_connection() as conn: - with conn.cursor() as cur: - cur.execute("SELECT id, name, email FROM customers WHERE email = %s", (email,)) - (cid, name, email) = cur.fetchone() - return Customer(cid, name, email) - - -def delete_all_customers(): - with get_connection() as conn: - with conn.cursor() as cur: - cur.execute("DELETE FROM customers") - conn.commit() -``` - -> [!NOTE] -> To keep it straightforward for this guide, each function creates a new -> connection. In a real-world application, use a connection pool to reuse -> connections. diff --git a/content/guides/testcontainers-python-getting-started/run-tests.md b/content/guides/testcontainers-python-getting-started/run-tests.md deleted file mode 100644 index 427756bb6b98..000000000000 --- a/content/guides/testcontainers-python-getting-started/run-tests.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -title: Run tests and next steps -linkTitle: Run tests -description: Run your Testcontainers-based integration tests and explore next steps. -keywords: testcontainers, python, postgresql, integration testing, run tests -weight: 30 ---- - -## Run the tests - -Run the tests using pytest: - -```console -$ pytest -v -``` - -You should see output similar to: - -```text -============================= test session starts ============================== -platform linux -- Python 3.13.x, pytest-9.x.x -collected 2 items - -tests/test_customers.py::test_get_all_customers PASSED [ 50%] -tests/test_customers.py::test_get_customer_by_email PASSED [100%] - -============================== 2 passed in 1.90s =============================== -``` - -The tests run against a real PostgreSQL database instead of mocks, which gives -more confidence in the implementation. - -## Summary - -The Testcontainers for Python library helps you write integration tests using the -same type of database (Postgres) that you use in production, instead of mocks. -Because you aren't using mocks and instead talk to real services, you're free -to refactor code and still verify that the application works as expected. - -In addition to PostgreSQL, Testcontainers for Python provides modules for many -SQL databases, NoSQL databases, messaging queues, and more. You can use -Testcontainers to run any containerized dependency for your tests. - -To learn more about Testcontainers, visit the -[Testcontainers overview](https://testcontainers.com/getting-started/). - -## Further reading - -- [testcontainers-python documentation](https://testcontainers-python.readthedocs.io/) -- [Getting started with Testcontainers for Go](/guides/testcontainers-go-getting-started/) -- [Getting started with Testcontainers for Java](https://testcontainers.com/guides/getting-started-with-testcontainers-for-java/) -- [Getting started with Testcontainers for Node.js](https://testcontainers.com/guides/getting-started-with-testcontainers-for-nodejs/) diff --git a/content/guides/testcontainers-python-getting-started/write-tests.md b/content/guides/testcontainers-python-getting-started/write-tests.md deleted file mode 100644 index 870c0d97f2d7..000000000000 --- a/content/guides/testcontainers-python-getting-started/write-tests.md +++ /dev/null @@ -1,105 +0,0 @@ ---- -title: Write tests with Testcontainers -linkTitle: Write tests -description: Write integration tests using testcontainers-python and pytest with a real PostgreSQL database. -keywords: testcontainers, python, postgresql, pytest, integration testing -weight: 20 ---- - -You'll create a PostgreSQL container using Testcontainers and use it for all -the tests. Before each test, you'll delete all customer records so that tests -run with a clean database. - -## Set up pytest fixtures - -This guide uses [pytest fixtures](https://pytest.org/en/stable/how-to/fixtures.html) -for setup and teardown logic. A recommended approach is to use -[finalizers](https://pytest.org/en/stable/how-to/fixtures.html#adding-finalizers-directly) -to guarantee cleanup runs even if setup fails: - -```python -@pytest.fixture -def setup(request): - # setup code - - def cleanup(): - # teardown code - - request.addfinalizer(cleanup) - return some_value -``` - -## Create the test file - -Create a `tests/__init__.py` file with empty content to enable pytest -[auto-discovery](https://pytest.org/explanation/goodpractices.html#test-discovery). - -Then create `tests/test_customers.py` with the fixtures: - -```python -import os -import pytest -from testcontainers.postgres import PostgresContainer - -from customers import customers - -postgres = PostgresContainer("postgres:16-alpine") - - -@pytest.fixture(scope="module", autouse=True) -def setup(request): - postgres.start() - - def remove_container(): - postgres.stop() - - request.addfinalizer(remove_container) - os.environ["DB_CONN"] = postgres.get_connection_url() - os.environ["DB_HOST"] = postgres.get_container_host_ip() - os.environ["DB_PORT"] = str(postgres.get_exposed_port(5432)) - os.environ["DB_USERNAME"] = postgres.username - os.environ["DB_PASSWORD"] = postgres.password - os.environ["DB_NAME"] = postgres.dbname - customers.create_table() - - -@pytest.fixture(scope="function", autouse=True) -def setup_data(): - customers.delete_all_customers() -``` - -Here's what the fixtures do: - -- The `setup` fixture has `scope="module"`, so it runs once for all tests in - the file. It starts a PostgreSQL container, sets environment variables with - the connection details, and creates the `customers` table. A cleanup - function removes the container after all tests complete. -- The `setup_data` fixture has `scope="function"`, so it runs before every - test. It deletes all records to give each test a clean database. - -## Write the tests - -Add the test functions to the same file: - -```python -def test_get_all_customers(): - customers.create_customer("Siva", "siva@gmail.com") - customers.create_customer("James", "james@gmail.com") - customers_list = customers.get_all_customers() - assert len(customers_list) == 2 - - -def test_get_customer_by_email(): - customers.create_customer("John", "john@gmail.com") - customer = customers.get_customer_by_email("john@gmail.com") - assert customer.name == "John" - assert customer.email == "john@gmail.com" -``` - -- `test_get_all_customers()` inserts two customer records, fetches all - customers, and asserts the count. -- `test_get_customer_by_email()` inserts a customer, fetches it by email, and - asserts the details. - -Because `setup_data` deletes all records before each test, the tests can run in -any order. diff --git a/content/guides/text-classification.md b/content/guides/text-classification.md index 81df374797a9..c57ba6261786 100644 --- a/content/guides/text-classification.md +++ b/content/guides/text-classification.md @@ -6,11 +6,10 @@ description: Learn how to build and run a text recognition application using Pyt summary: | This guide details how to containerize text classification models using Docker. -tags: [ai] -languages: [python] aliases: - /guides/use-case/nlp/text-classification/ params: + tags: [ai] time: 20 minutes --- diff --git a/content/guides/text-summarization.md b/content/guides/text-summarization.md index e4657f072b12..a0409e862c63 100644 --- a/content/guides/text-summarization.md +++ b/content/guides/text-summarization.md @@ -5,11 +5,10 @@ keywords: nlp, natural language processing, text summarization, python, bert ext description: Learn how to build and run a text summarization application using Python, Bert Extractive Summarizer, and Docker. summary: | This guide shows how to containerize text summarization models with Docker. -tags: [ai] -languages: [python] aliases: - /guides/use-case/nlp/text-summarization/ params: + tags: [ai] time: 20 minutes --- diff --git a/content/guides/traefik.md b/content/guides/traefik.md index 9130763bc669..c91c42032935 100644 --- a/content/guides/traefik.md +++ b/content/guides/traefik.md @@ -4,8 +4,8 @@ description: &desc Use Traefik to easily route traffic between multiple containe keywords: traefik, container-supported development linktitle: HTTP routing with Traefik summary: *desc -tags: [networking, dhi] params: + tags: [deployment] time: 20 minutes --- diff --git a/content/guides/vuejs/_index.md b/content/guides/vuejs/_index.md index 8bd306da6c4b..3db4bc43e261 100644 --- a/content/guides/vuejs/_index.md +++ b/content/guides/vuejs/_index.md @@ -5,17 +5,19 @@ description: Containerize and develop Vue.js apps using Docker keywords: getting started, vue, vuejs docker, language, Dockerfile summary: | This guide explains how to containerize Vue.js applications using Docker. -toc_min: 1 -toc_max: 2 -languages: [js] -tags: [frameworks, dhi] aliases: - /frameworks/vue/ + - /guides/vuejs/configure-github-actions/ + - /guides/vuejs/containerize/ + - /guides/vuejs/deploy/ + - /guides/vuejs/develop/ + - /guides/vuejs/run-tests/ params: + tags: [cicd] time: 20 minutes - --- + The Vue.js language-specific guide shows you how to containerize an Vue.js application using Docker, following best practices for creating efficient, production-ready containers. [Vue.js](https://vuejs.org/) is a progressive and flexible framework for building modern, interactive web applications. However, as applications scale, managing dependencies, environments, and deployments can become complex. Docker simplifies these challenges by providing a consistent, isolated environment for both development and production. @@ -49,4 +51,1350 @@ Before you begin, ensure you have a working knowledge of: - Familiarity with [Vue.js](https://vuejs.org/) fundamentals. - Understanding of core Docker concepts such as images, containers, and Dockerfiles. If you're new to Docker, start with the [Docker basics](/get-started/docker-concepts/the-basics/what-is-a-container.md) guide. -Once you've completed the Vue.js getting started modules, you’ll be fully prepared to containerize your own Vue.js application using the detailed examples and best practices outlined in this guide. \ No newline at end of file +Once you've completed the Vue.js getting started modules, you’ll be fully prepared to containerize your own Vue.js application using the detailed examples and best practices outlined in this guide. + +## Containerize an Vue.js Application + +### Prerequisites + +Before you begin, make sure the following tools are installed and available on your system: + +- You have installed the latest version of [Docker Desktop](/get-started/get-docker.md). +- You have a [git client](https://git-scm.com/downloads). The examples in this section use a command-line based git client, but you can use any client. + +> **New to Docker?** +> Start with the [Docker basics](/get-started/docker-concepts/the-basics/what-is-a-container.md) guide to get familiar with key concepts like images, containers, and Dockerfiles. + +--- + +### Overview + +This guide walks you through the complete process of containerizing an Vue.js application with Docker. You’ll learn how to create a production-ready Docker image using best practices that improve performance, security, scalability, and deployment efficiency. + +By the end of this guide, you will: + +- Containerize an Vue.js application using Docker. +- Create and optimize a Dockerfile for production builds. +- Use multi-stage builds to minimize image size. +- Serve the application efficiently with a custom Nginx configuration. +- Build secure and maintainable Docker images by following best practices. + +--- + +### Get the sample application + +Clone the sample application to use with this guide. Open a terminal, navigate to the directory where you want to work, and run the following command +to clone the git repository: + +```console +$ git clone https://github.com/kristiyan-velkov/docker-vuejs-sample +``` +--- + +### Build the Docker image + +Vue.js is a front-end framework that compiles into static assets, so the Dockerfile is customized to align with how Vue.js applications are built and efficiently served in a production environment. + +> [!TIP] +> +> [Gordon](/ai/gordon/), Docker's AI assistant, can generate Docker assets for your project. Ask Gordon to create a Dockerfile, Compose file, and `.dockerignore` tailored to your application. + +#### Step 1: Create the Dockerfile + +Before creating a Dockerfile, you need to choose a base image. You can either use the [Node.js Official Image](https://hub.docker.com/_/node) or a Docker Hardened Image (DHI) from the [Hardened Image catalog](https://hub.docker.com/hardened-images/catalog). + +Choosing DHI offers the advantage of a production-ready image that is lightweight and secure. For more information, see [Docker Hardened Images](https://docs.docker.com/dhi/). + +> [!IMPORTANT] +> This guide uses a stable Node.js LTS image tag that is considered secure when the guide is written. Because new releases and security patches are published regularly, the tag shown here may no longer be the safest option when you follow the guide. Always review the latest available image tags and select a secure, up-to-date version before building or deploying your application. +> +> Official Node.js Docker Images: https://hub.docker.com/_/node + +{{< tabs >}} +{{< tab name="Using Docker Hardened Images" >}} +Docker Hardened Images (DHIs) are available for Node.js in the [Docker Hardened Images catalog](https://hub.docker.com/hardened-images/catalog/dhi/node). Docker Hardened Images are freely available to everyone with no subscription required. You can pull and use them like any other Docker image after signing in to the DHI registry. For more information, see the [DHI quickstart](/dhi/get-started/) guide. + +1. Sign in to the DHI registry: + ```console + $ docker login dhi.io + ``` + +2. Pull the Node.js DHI (check the catalog for available versions): + ```console + $ docker pull dhi.io/node:24-alpine3.22-dev + ``` + +3. Pull the Nginx DHI (check the catalog for available versions): + ```console + $ docker pull dhi.io/nginx:1.28.0-alpine3.21-dev + ``` + +In the following Dockerfile, the `FROM` instructions use `dhi.io/node:24-alpine3.22-dev` and `dhi.io/nginx:1.28.0-alpine3.21-dev` as the base images. + +```dockerfile +# ========================================= +# Stage 1: Build the Vue.js Application +# ========================================= +# Use a lightweight DHI Node.js image for building +FROM dhi.io/node:24-alpine3.22-dev AS builder + +# Set the working directory inside the container +WORKDIR /app + +# Copy package-related files first to leverage Docker's caching mechanism +COPY package.json package-lock.json* ./ + +# Install project dependencies using npm ci (ensures a clean, reproducible install) +RUN --mount=type=cache,target=/root/.npm npm ci + +# Copy the rest of the application source code into the container +COPY . . + +# Build the Vue.js application +RUN npm run build + +# ========================================= +# Stage 2: Prepare Nginx to Serve Static Files +# ========================================= + +FROM dhi.io/nginx:1.28.0-alpine3.21-dev AS runner + +# Copy custom Nginx config +COPY nginx.conf /etc/nginx/nginx.conf + +# Copy the static build output from the build stage to Nginx's default HTML serving directory +COPY --chown=nginx:nginx --from=builder /app/dist /usr/share/nginx/html + +# Use a built-in non-root user for security best practices +USER nginx + +# Expose port 8080 to allow HTTP traffic +# Note: The default Nginx container now listens on port 8080 instead of 80 +EXPOSE 8080 + +# Start Nginx directly with custom config +ENTRYPOINT ["nginx", "-c", "/etc/nginx/nginx.conf"] +CMD ["-g", "daemon off;"] +``` + +{{< /tab >}} +{{< tab name="Using the Docker Official Image" >}} + +Create a file named `Dockerfile` with the following contents: + +```dockerfile +# ========================================= +# Stage 1: Build the Vue.js Application +# ========================================= +ARG NODE_VERSION=24.12.0-alpine +ARG NGINX_VERSION=alpine3.22 + +# Use a lightweight Node.js image for building (customizable via ARG) +FROM node:${NODE_VERSION} AS builder + +# Set the working directory inside the container +WORKDIR /app + +# Copy package-related files first to leverage Docker's caching mechanism +COPY package.json package-lock.json* ./ + +# Install project dependencies using npm ci (ensures a clean, reproducible install) +RUN --mount=type=cache,target=/root/.npm npm ci + +# Copy the rest of the application source code into the container +COPY . . + +# Build the Vue.js application +RUN npm run build + +# ========================================= +# Stage 2: Prepare Nginx to Serve Static Files +# ========================================= + +FROM nginxinc/nginx-unprivileged:${NGINX_VERSION} AS runner + +# Copy custom Nginx config +COPY nginx.conf /etc/nginx/nginx.conf + +# Copy the static build output from the build stage to Nginx's default HTML serving directory +COPY --chown=nginx:nginx --from=builder /app/dist /usr/share/nginx/html + +# Use a built-in non-root user for security best practices +USER nginx + +# Expose port 8080 to allow HTTP traffic +# Note: The default Nginx container now listens on port 8080 instead of 80 +EXPOSE 8080 + +# Start Nginx directly with custom config +ENTRYPOINT ["nginx", "-c", "/etc/nginx/nginx.conf"] +CMD ["-g", "daemon off;"] +``` + +> [!NOTE] +> We are using nginx-unprivileged instead of the standard Nginx image to follow security best practices. +> Running as a non-root user in the final image: +>- Reduces the attack surface +>- Aligns with Docker’s recommendations for container hardening +>- Helps comply with stricter security policies in production environments + +{{< /tab >}} +{{< /tabs >}} + +#### Step 2: Create the compose.yaml file + +Create a file named `compose.yaml` with the following contents: + +```yaml {collapse=true,title=compose.yaml} +services: + server: + build: + context: . + ports: + - 8080:8080 +``` + +#### Step 3: Create the .dockerignore file + +The `.dockerignore` file plays a crucial role in optimizing your Docker image by specifying which files and directories should be excluded from the build context. + +> [!NOTE] +>This helps: +>- Reduce image size +>- Speed up the build process +>- Prevent sensitive or unnecessary files (like `.env`, `.git`, or `node_modules`) from being added to the final image. +> +> To learn more, visit the [.dockerignore reference](/reference/dockerfile.md#dockerignore-file). + +Create a file named `.dockerignore` with the following contents: + +```dockerignore +# ------------------------------- +# Dependency directories +# ------------------------------- +node_modules/ + +# ------------------------------- +# Production and build outputs +# ------------------------------- +dist/ +out/ +build/ +public/build/ + +# ------------------------------- +# Vite, VuePress, and cache dirs +# ------------------------------- +.vite/ +.vitepress/ +.cache/ +.tmp/ + +# ------------------------------- +# Test output and coverage +# ------------------------------- +coverage/ +reports/ +jest/ +cypress/ +cypress/screenshots/ +cypress/videos/ + +# ------------------------------- +# Environment and config files +# ------------------------------- +*.env* +!.env.production # Keep production env if needed +*.local +*.log + +# ------------------------------- +# TypeScript artifacts +# ------------------------------- +*.tsbuildinfo + +# ------------------------------- +# Editor and IDE config +# ------------------------------- +.vscode/ +.idea/ +*.swp + +# ------------------------------- +# System files +# ------------------------------- +.DS_Store +Thumbs.db + +# ------------------------------- +# Lockfiles (optional) +# ------------------------------- +npm-debug.log* +yarn-debug.log* +yarn-error.log* +pnpm-debug.log* + +# ------------------------------- +# Git files +# ------------------------------- +.git/ +.gitignore + +# ------------------------------- +# Docker-related files +# ------------------------------- +Dockerfile +.dockerignore +docker-compose.yml +docker-compose.override.yml +``` + +#### Step 4: Create the `nginx.conf` file + +To serve your Vue.js application efficiently inside the container, you’ll configure Nginx with a custom setup. This configuration is optimized for performance, browser caching, gzip compression, and support for client-side routing. + +Create a file named `nginx.conf` in the root of your project directory, and add the following content: + +> [!NOTE] +> To learn more about configuring Nginx, see the [official Nginx documentation](https://nginx.org/en/docs/). + +```nginx +worker_processes auto; +pid /tmp/nginx.pid; + +events { + worker_connections 1024; +} + +http { + include /etc/nginx/mime.types; + default_type application/octet-stream; + charset utf-8; + + access_log off; + error_log /dev/stderr warn; + + sendfile on; + tcp_nopush on; + tcp_nodelay on; + keepalive_timeout 65; + keepalive_requests 1000; + + gzip on; + gzip_comp_level 6; + gzip_proxied any; + gzip_min_length 256; + gzip_vary on; + gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript image/svg+xml; + + server { + listen 8080; + server_name localhost; + + root /usr/share/nginx/html; + index index.html; + + location / { + try_files $uri $uri/ /index.html; + } + + location ~* \.(?:ico|css|js|gif|jpe?g|png|woff2?|eot|ttf|svg|map)$ { + expires 1y; + access_log off; + add_header Cache-Control "public, immutable"; + add_header X-Content-Type-Options nosniff; + } + + location /assets/ { + expires 1y; + add_header Cache-Control "public, immutable"; + add_header X-Content-Type-Options nosniff; + } + + error_page 404 /index.html; + } +} +``` + +#### Step 5: Build the Vue.js application image + +With your custom configuration in place, you're now ready to build the Docker image for your Vue.js application. + +The updated setup includes: + +- The updated setup includes a clean, production-ready Nginx configuration tailored specifically for Vue.js. +- Efficient multi-stage Docker build, ensuring a small and secure final image. + +After completing the previous steps, your project directory should now contain the following files: + +```text +├── docker-vuejs-sample/ +│ ├── Dockerfile +│ ├── .dockerignore +│ ├── compose.yaml +│ └── nginx.conf +``` + +Now that your Dockerfile is configured, you can build the Docker image for your Vue.js application. + +> [!NOTE] +> The `docker build` command packages your application into an image using the instructions in the Dockerfile. It includes all necessary files from the current directory (called the [build context](/build/concepts/context/#what-is-a-build-context)). + +Run the following command from the root of your project: + +```console +$ docker build --tag docker-vuejs-sample . +``` + +What this command does: +- Uses the Dockerfile in the current directory (.) +- Packages the application and its dependencies into a Docker image +- Tags the image as docker-vuejs-sample so you can reference it later + + +#### Step 6: View local images + +After building your Docker image, you can check which images are available on your local machine using either the Docker CLI or [Docker Desktop](/manuals/desktop/use-desktop/images.md). Since you're already working in the terminal, let's use the Docker CLI. + +To list all locally available Docker images, run the following command: + +```console +$ docker images +``` + +Example Output: + +```shell +REPOSITORY TAG IMAGE ID CREATED SIZE +docker-vuejs-sample latest 8c9c199179d4 14 seconds ago 76.2MB +``` + +This output provides key details about your images: + +- **Repository** – The name assigned to the image. +- **Tag** – A version label that helps identify different builds (e.g., latest). +- **Image ID** – A unique identifier for the image. +- **Created** – The timestamp indicating when the image was built. +- **Size** – The total disk space used by the image. + +If the build was successful, you should see `docker-vuejs-sample` image listed. + +--- + +### Run the containerized application + +In the previous step, you created a Dockerfile for your Vue.js application and built a Docker image using the docker build command. Now it’s time to run that image in a container and verify that your application works as expected. + + +Inside the `docker-vuejs-sample` directory, run the following command in a +terminal. + +```console +$ docker compose up --build +``` + +Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see a simple Vue.js web application. + +Press `ctrl+c` in the terminal to stop your application. + +#### Run the application in the background + +You can run the application detached from the terminal by adding the `-d` +option. Inside the `docker-vuejs-sample` directory, run the following command +in a terminal. + +```console +$ docker compose up --build -d +``` + +Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see your Vue.js application running in the browser. + + +To confirm that the container is running, use `docker ps` command: + +```console +$ docker ps +``` + +This will list all active containers along with their ports, names, and status. Look for a container exposing port 8080. + +Example Output: + +```shell +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +37a1fa85e4b0 docker-vuejs-sample-server "nginx -c /etc/nginx…" About a minute ago Up About a minute 0.0.0.0:8080->8080/tcp docker-vuejs-sample-server-1 +``` + + +To stop the application, run: + +```console +$ docker compose down +``` + + +> [!NOTE] +> For more information about Compose commands, see the [Compose CLI +> reference](/reference/cli/docker/compose/). + +--- + +### Summary + +In this guide, you learned how to containerize, build, and run an Vue.js application using Docker. By following best practices, you created a secure, optimized, and production-ready setup. + +What you accomplished: +- Created a multi-stage `Dockerfile` that compiles the Vue.js application and serves the static files using Nginx. +- Created a `.dockerignore` file to exclude unnecessary files and keep the image clean and efficient. +- Built your Docker image using `docker build`. +- Ran the container using `docker compose up`, both in the foreground and in detached mode. +- Verified that the app was running by visiting [http://localhost:8080](http://localhost:8080). +- Learned how to stop the containerized application using `docker compose down`. + +You now have a fully containerized Vue.js application, running in a Docker container, and ready for deployment across any environment with confidence and consistency. + +--- + +### Related resources + +Explore official references and best practices to sharpen your Docker workflow: + +- [Multi-stage builds](/build/building/multi-stage/) – Learn how to separate build and runtime stages. +- [Best practices for writing Dockerfiles](/develop/develop-images/dockerfile_best-practices/) – Write efficient, maintainable, and secure Dockerfiles. +- [Build context in Docker](/build/concepts/context/) – Learn how context affects image builds. +- [`docker build` CLI reference](/reference/cli/docker/image/build/) – Build Docker images from a Dockerfile. +- [`docker images` CLI reference](/reference/cli/docker/image/ls/) – Manage and inspect local Docker images. +- [`docker compose up` CLI reference](/reference/cli/docker/compose/up/) – Start and run multi-container applications. +- [`docker compose down` CLI reference](/reference/cli/docker/compose/down/) – Stop and remove containers, networks, and volumes. + +--- + +### Next steps + +With your Vue.js application now containerized, you're ready to move on to the next step. + +In the next section, you'll learn how to develop your application using Docker containers, enabling a consistent, isolated, and reproducible development environment across any machine. + +## Use containers for Vue.js development + +### Prerequisites + +Complete [Containerize Vue.js application](containerize.md). + +--- + +### Overview + +In this section, you'll set up both production and development environments for your Vue.js application using Docker Compose. This approach streamlines your workflow—delivering a lightweight, static site via Nginx in production, and providing a fast, live-reloading dev server with Compose Watch for efficient local development. + +You’ll learn how to: +- Configure isolated environments: Set up separate containers optimized for production and development use cases. +- Live-reload in development: Use Compose Watch to automatically sync file changes, enabling real-time updates without manual intervention. +- Preview and debug with ease: Develop inside containers with a seamless preview and debug experience—no rebuilds required after every change. + +--- + +### Automatically update services (development mode) + +Leverage Compose Watch to enable real-time file synchronization between your local machine and the containerized Vue.js development environment. This powerful feature eliminates the need to manually rebuild or restart containers, providing a fast, seamless, and efficient development workflow. + +With Compose Watch, your code updates are instantly reflected inside the container—perfect for rapid testing, debugging, and live previewing changes. + +### Step 1: Create a development Dockerfile + +Create a file named `Dockerfile.dev` in your project root with the following content: + +```dockerfile +# ========================================= +# Stage 1: Develop the Vue.js Application +# ========================================= +ARG NODE_VERSION=24.12.0-alpine + +# Use a lightweight Node.js image for development +FROM node:${NODE_VERSION} AS dev + +# Set environment variable to indicate development mode +ENV NODE_ENV=development + +# Set the working directory inside the container +WORKDIR /app + +# Copy package-related files first to leverage Docker's caching mechanism +COPY package.json package-lock.json* ./ + +# Install project dependencies +RUN --mount=type=cache,target=/root/.npm npm install + +# Copy the rest of the application source code into the container +COPY . . + +# Change ownership of the application directory to the node user +RUN chown -R node:node /app + +# Switch to the node user +USER node + +# Expose the port used by the Vite development server +EXPOSE 5173 + +# Use a default command, can be overridden in Docker compose.yml file +CMD [ "npm", "run", "dev", "--", "--host" ] + +``` + +This file sets up a lightweight development environment for your Vue.js application using the dev server. + +#### Step 2: Update your `compose.yaml` file + +Open your `compose.yaml` file and define two services: one for production (`vuejs-prod`) and one for development (`vuejs-dev`). + +Here’s an example configuration for an Vue.js application: + +```yaml +services: + vuejs-prod: + build: + context: . + dockerfile: Dockerfile + image: docker-vuejs-sample + ports: + - "8080:8080" + + vuejs-dev: + build: + context: . + dockerfile: Dockerfile.dev + ports: + - "5173:5173" + develop: + watch: + - path: ./src + target: /app/src + action: sync + - path: ./package.json + target: /app/package.json + action: restart + - path: ./vite.config.js + target: /app/vite.config.js + action: restart +``` +- The `vuejs-prod` service builds and serves your static production app using Nginx. +- The `vuejs-dev` service runs your Vue.js development server with live reload and hot module replacement. +- `watch` triggers file sync with Compose Watch. + +> [!NOTE] +> For more details, see the official guide: [Use Compose Watch](/manuals/compose/how-tos/file-watch.md). + +After completing the previous steps, your project directory should now contain the following files: + +```text +├── docker-vuejs-sample/ +│ ├── Dockerfile +│ ├── Dockerfile.dev +│ ├── .dockerignore +│ ├── compose.yaml +│ └── nginx.conf +``` + +#### Step 4: Start Compose Watch + +Run the following command from the project root to start the container in watch mode + +```console +$ docker compose watch vuejs-dev +``` + +#### Step 5: Test Compose Watch with Vue.js + +To confirm that Compose Watch is functioning correctly: + +1. Open the `src/App.vue` file in your text editor. + +2. Locate the following line: + + ```html + + ``` + +3. Change it to: + + ```html + + ``` + +4. Save the file. + +5. Open your browser at [http://localhost:5173](http://localhost:5173). + +You should see the updated text appear instantly, without needing to rebuild the container manually. This confirms that file watching and automatic synchronization are working as expected. + +--- + +### Summary + +In this section, you set up a complete development and production workflow for your Vue.js application using Docker and Docker Compose. + +Here’s what you accomplished: +- Created a `Dockerfile.dev` to streamline local development with hot reloading +- Defined separate `vuejs-dev` and `vuejs-prod` services in your `compose.yaml` file +- Enabled real-time file syncing using Compose Watch for a smoother development experience +- Verified that live updates work seamlessly by modifying and previewing a component + +With this setup, you're now equipped to build, run, and iterate on your Vue.js app entirely within containers—efficiently and consistently across environments. + +--- + +### Related resources + +Deepen your knowledge and improve your containerized development workflow with these guides: + +- [Using Compose Watch](/manuals/compose/how-tos/file-watch.md) – Automatically sync source changes during development +- [Multi-stage builds](/manuals/build/building/multi-stage.md) – Create efficient, production-ready Docker images +- [Dockerfile best practices](/build/building/best-practices/) – Write clean, secure, and optimized Dockerfiles. +- [Compose file reference](/compose/compose-file/) – Learn the full syntax and options available for configuring services in `compose.yaml`. +- [Docker volumes](/storage/volumes/) – Persist and manage data between container runs + +### Next steps + +In the next section, you'll learn how to run unit tests for your Vue.js application inside Docker containers. This ensures consistent testing across all environments and removes dependencies on local machine setup. + +## Run vue.js tests in a container + +### Prerequisites + +Complete all the previous sections of this guide, starting with [Containerize Vue.js application](containerize.md). + +### Overview + +Testing is a critical part of the development process. In this section, you'll learn how to: + +- Run unit tests using Vitest inside a Docker container. +- Use Docker Compose to run tests in an isolated, reproducible environment. + +You’ll use [Vitest](https://vitest.dev) — a blazing fast test runner designed for Vite — together with [@vue/test-utils](https://test-utils.vuejs.org/) to write unit tests that validate your component logic, props, events, and reactive behavior. + +This setup ensures your Vue.js components are tested in an environment that mirrors how users actually interact with your application. + +--- + +### Run tests during development + +`docker-vuejs-sample` application includes a sample test file at location: + +```console +$ src/components/__tests__/HelloWorld.spec.ts +``` + +This test uses Vitest and Vue Test Utils to verify the behavior of the HelloWorld component. + +--- + +#### Step 1: Update compose.yaml + +Add a new service named `vuejs-test` to your `compose.yaml` file. This service allows you to run your test suite in an isolated containerized environment. + +```yaml {hl_lines="22-26",linenos=true} +services: + vuejs-prod: + build: + context: . + dockerfile: Dockerfile + image: docker-vuejs-sample + ports: + - "8080:8080" + + vuejs-dev: + build: + context: . + dockerfile: Dockerfile.dev + ports: + - "5173:5173" + develop: + watch: + - action: sync + path: . + target: /app + + vuejs-test: + build: + context: . + dockerfile: Dockerfile.dev + command: ["npm", "run", "test:unit"] +``` + +The vuejs-test service reuses the same `Dockerfile.dev` used for [development](develop.md) and overrides the default command to run tests with `npm run test`. This setup ensures a consistent test environment that matches your local development configuration. + + +After completing the previous steps, your project directory should contain the following files: + +```text +├── docker-vuejs-sample/ +│ ├── Dockerfile +│ ├── Dockerfile.dev +│ ├── .dockerignore +│ ├── compose.yaml +│ └── nginx.conf +``` + +#### Step 2: Run the tests + +To execute your test suite inside the container, run the following command from your project root: + +```console +$ docker compose run --rm vuejs-test +``` + +This command will: +- Start the `vuejs-test` service defined in your `compose.yaml` file. +- Execute the `npm run test` script using the same environment as development. +- Automatically remove the container after the tests complete [`docker compose run --rm`](/reference/cli/docker/compose/run/) command. + +You should see output similar to the following: + +```shell +Test Files: 1 passed (1) +Tests: 1 passed (1) +Start at: 16:50:55 +Duration: 718ms +``` + +> [!NOTE] +> For more information about Compose commands, see the [Compose CLI +> reference](/reference/cli/docker/compose/). + +--- + +### Summary + +In this section, you learned how to run unit tests for your Vue.js application inside a Docker container using Vitest and Docker Compose. + +What you accomplished: +- Created a `vuejs-test` service in `compose.yaml` to isolate test execution. +- Reused the development `Dockerfile.dev` to ensure consistency between dev and test environments. +- Ran tests inside the container using `docker compose run --rm vuejs-test`. +- Ensured reliable, repeatable testing across environments without depending on your local machine setup. + +--- + +### Related resources + +Explore official references and best practices to sharpen your Docker testing workflow: + +- [Dockerfile reference](/reference/dockerfile/) – Understand all Dockerfile instructions and syntax. +- [Best practices for writing Dockerfiles](/develop/develop-images/dockerfile_best-practices/) – Write efficient, maintainable, and secure Dockerfiles. +- [Compose file reference](/compose/compose-file/) – Learn the full syntax and options available for configuring services in `compose.yaml`. +- [`docker compose run` CLI reference](/reference/cli/docker/compose/run/) – Run one-off commands in a service container. +--- + +### Next steps + +Next, you’ll learn how to set up a CI/CD pipeline using GitHub Actions to automatically build and test your Vue.js application in a containerized environment. This ensures your code is validated on every push or pull request, maintaining consistency and reliability across your development workflow. + +## Test your Vue.js deployment + +### Prerequisites + +Before you begin, make sure you’ve completed the following: +- Complete all the previous sections of this guide, starting with [Containerize Vue.js application](containerize.md). +- [Enable Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. + +> **New to Kubernetes?** +> Visit the [Kubernetes basics tutorial](https://kubernetes.io/docs/tutorials/kubernetes-basics/) to get familiar with how clusters, pods, deployments, and services work. + +--- + +### Overview + +This section guides you through deploying your containerized Vue.js application locally using [Docker Desktop’s built-in Kubernetes](/desktop/kubernetes/). Running your app in a local Kubernetes cluster closely simulates a real production environment, enabling you to test, validate, and debug your workloads with confidence before promoting them to staging or production. + +--- + +### Create a Kubernetes YAML file + +Follow these steps to define your deployment configuration: + +1. In the root of your project, create a new file named: vuejs-sample-kubernetes.yaml + +2. Open the file in your IDE or preferred text editor. + +3. Add the following configuration, and be sure to replace `{DOCKER_USERNAME}` and `{DOCKERHUB_PROJECT_NAME}` with your actual Docker Hub username and repository name from the previous [Automate your builds with GitHub Actions](configure-github-actions.md). + + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: vuejs-sample + namespace: default +spec: + replicas: 1 + selector: + matchLabels: + app: vuejs-sample + template: + metadata: + labels: + app: vuejs-sample + spec: + containers: + - name: vuejs-container + image: {DOCKER_USERNAME}/{DOCKERHUB_PROJECT_NAME}:latest + imagePullPolicy: Always + ports: + - containerPort: 8080 + resources: + limits: + cpu: "500m" + memory: "256Mi" + requests: + cpu: "250m" + memory: "128Mi" +--- +apiVersion: v1 +kind: Service +metadata: + name: vuejs-sample-service + namespace: default +spec: + type: NodePort + selector: + app: vuejs-sample + ports: + - port: 8080 + targetPort: 8080 + nodePort: 30001 +``` + +This manifest defines two key Kubernetes resources, separated by `---`: + +- Deployment + Deploys a single replica of your Vue.js application inside a pod. The pod uses the Docker image built and pushed by your GitHub Actions CI/CD workflow + (refer to [Automate your builds with GitHub Actions](configure-github-actions.md)). + The container listens on port `8080`, which is typically used by [Nginx](https://nginx.org/en/docs/) to serve your production Vue.js app. + +- Service (NodePort) + Exposes the deployed pod to your local machine. + It forwards traffic from port `30001` on your host to port `8080` inside the container. + This lets you access the application in your browser at [http://localhost:30001](http://localhost:30001). + +> [!NOTE] +> To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). + +--- + +### Deploy and check your application + +Follow these steps to deploy your containerized Vue.js app into a local Kubernetes cluster and verify that it’s running correctly. + +#### Step 1. Apply the Kubernetes configuration + +In your terminal, navigate to the directory where your `vuejs-sample-kubernetes.yaml` file is located, then deploy the resources using: + +```console + $ kubectl apply -f vuejs-sample-kubernetes.yaml +``` + +If everything is configured properly, you’ll see confirmation that both the Deployment and the Service were created: + +```shell + deployment.apps/vuejs-sample created + service/vuejs-sample-service created +``` + +This confirms that both the Deployment and the Service were successfully created and are now running inside your local cluster. + +#### Step 2. Check the deployment status + +Run the following command to check the status of your deployment: + +```console + $ kubectl get deployments +``` + +You should see output similar to the following: + +```shell + NAME READY UP-TO-DATE AVAILABLE AGE + vuejs-sample 1/1 1 1 1m14s +``` + +This confirms that your pod is up and running with one replica available. + +#### Step 3. Verify the service exposure + +Check if the NodePort service is exposing your app to your local machine: + +```console +$ kubectl get services +``` + +You should see something like: + +```shell +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +vuejs-sample-service NodePort 10.98.233.59 8080:30001/TCP 1m +``` + +This output confirms that your app is available via NodePort on port 30001. + +#### Step 4. Access your app in the browser + +Open your browser and navigate to [http://localhost:30001](http://localhost:30001). + +You should see your production-ready Vue.js Sample application running — served by your local Kubernetes cluster. + +#### Step 5. Clean up Kubernetes resources + +Once you're done testing, you can delete the deployment and service using: + +```console + $ kubectl delete -f vuejs-sample-kubernetes.yaml +``` + +Expected output: + +```shell + deployment.apps "vuejs-sample" deleted + service "vuejs-sample-service" deleted +``` + +This ensures your cluster stays clean and ready for the next deployment. + +--- + +### Summary + +In this section, you learned how to deploy your Vue.js application to a local Kubernetes cluster using Docker Desktop. This setup allows you to test and debug your containerized app in a production-like environment before deploying it to the cloud. + +What you accomplished: + +- Created a Kubernetes Deployment and NodePort Service for your Vue.js app +- Used `kubectl apply` to deploy the application locally +- Verified the app was running and accessible at `http://localhost:30001` +- Cleaned up your Kubernetes resources after testing + +--- + +### Related resources + +Explore official references and best practices to sharpen your Kubernetes deployment workflow: + +- [Kubernetes documentation](https://kubernetes.io/docs/home/) – Learn about core concepts, workloads, services, and more. +- [Deploy on Kubernetes with Docker Desktop](/manuals) – Use Docker Desktop’s built-in Kubernetes support for local testing and development. +- [`kubectl` CLI reference](https://kubernetes.io/docs/reference/kubectl/) – Manage Kubernetes clusters from the command line. +- [Kubernetes Deployment resource](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) – Understand how to manage and scale applications using Deployments. +- [Kubernetes Service resource](https://kubernetes.io/docs/concepts/services-networking/service/) – Learn how to expose your application to internal and external traffic. + +## Automate your builds with GitHub Actions + +### Prerequisites + +Complete all the previous sections of this guide, starting with [Containerize an Vue.js application](containerize.md). + +You must also have: +- A [GitHub](https://github.com/signup) account. +- A verified [Docker Hub](https://hub.docker.com/signup) account. + +--- + +### Overview + +In this section, you'll set up a CI/CD pipeline using [GitHub Actions](https://docs.github.com/en/actions) to automatically: + +- Build your Vue.js application inside a Docker container. +- Run tests in a consistent environment. +- Push the production-ready image to [Docker Hub](https://hub.docker.com). + +--- + +### Connect your GitHub repository to Docker Hub + +To enable GitHub Actions to build and push Docker images, you’ll securely store your Docker Hub credentials in your new GitHub repository. + +#### Step 1: Generate Docker Hub credentials and set GitHub secrets + +1. Create a Personal Access Token (PAT) from [Docker Hub](https://hub.docker.com) + 1. Go to your **Docker Hub account → Account Settings → Security**. + 2. Generate a new Access Token with **Read/Write** permissions. + 3. Name it something like `docker-vuejs-sample`. + 4. Copy and save the token — you’ll need it in Step 4. + +2. Create a repository in [Docker Hub](https://hub.docker.com/repositories/) + 1. Go to your **Docker Hub account → Create a repository**. + 2. For the Repository Name, use something descriptive — for example: `vuejs-sample`. + 3. Once created, copy and save the repository name — you’ll need it in Step 4. + +3. Create a new [GitHub repository](https://github.com/new) for your Vue.js project + +4. Add Docker Hub credentials as GitHub repository secrets + + In your newly created GitHub repository: + + 1. Navigate to: + **Settings → Secrets and variables → Actions → New repository secret**. + + 2. Add the following secrets: + + | Name | Value | + |-------------------|--------------------------------| + | `DOCKER_USERNAME` | Your Docker Hub username | + | `DOCKERHUB_TOKEN` | Your Docker Hub access token (created in Step 1) | + | `DOCKERHUB_PROJECT_NAME` | Your Docker Project Name (created in Step 2) | + + These secrets allow GitHub Actions to authenticate securely with Docker Hub during automated workflows. + +5. Connect Your Local Project to GitHub + + Link your local project `docker-vuejs-sample` to the GitHub repository you just created by running the following command from your project root: + + ```console + $ git remote set-url origin https://github.com/{your-username}/{your-repository-name}.git + ``` + + >[!IMPORTANT] + >Replace `{your-username}` and `{your-repository}` with your actual GitHub username and repository name. + + To confirm that your local project is correctly connected to the remote GitHub repository, run: + + ```console + $ git remote -v + ``` + + You should see output similar to: + + ```console + origin https://github.com/{your-username}/{your-repository-name}.git (fetch) + origin https://github.com/{your-username}/{your-repository-name}.git (push) + ``` + + This confirms that your local repository is properly linked and ready to push your source code to GitHub. + +6. Push your source code to GitHub + + Follow these steps to commit and push your local project to your GitHub repository: + + 1. Stage all files for commit. + + ```console + $ git add -A + ``` + This command stages all changes — including new, modified, and deleted files — preparing them for commit. + + + 2. Commit the staged changes with a descriptive message. + + ```console + $ git commit -m "Initial commit" + ``` + This command creates a commit that snapshots the staged changes with a descriptive message. + + 3. Push the code to the `main` branch. + + ```console + $ git push -u origin main + ``` + This command pushes your local commits to the `main` branch of the remote GitHub repository and sets the upstream branch. + +Once completed, your code will be available on GitHub, and any GitHub Actions workflow you’ve configured will run automatically. + +> [!NOTE] +> Learn more about the Git commands used in this step: +> - [Git add](https://git-scm.com/docs/git-add) – Stage changes (new, modified, deleted) for commit +> - [Git commit](https://git-scm.com/docs/git-commit) – Save a snapshot of your staged changes +> - [Git push](https://git-scm.com/docs/git-push) – Upload local commits to your GitHub repository +> - [Git remote](https://git-scm.com/docs/git-remote) – View and manage remote repository URLs + +--- + +#### Step 2: Set up the workflow + +Now you'll create a GitHub Actions workflow that builds your Docker image, runs tests, and pushes the image to Docker Hub. + +1. Go to your repository on GitHub and select the **Actions** tab in the top menu. + +2. Select **Set up a workflow yourself** when prompted. + + This opens an inline editor to create a new workflow file. By default, it will be saved to: + `.github/workflows/main.yml` + + +3. Add the following workflow configuration to the new file: + +```yaml +name: CI/CD – Vue.js App with Docker + +on: + push: + branches: [main] + pull_request: + branches: [main] + types: [opened, synchronize, reopened] + +jobs: + build-test-deploy: + name: Build, Test & Deploy + runs-on: ubuntu-latest + + steps: + # 1. Checkout the codebase + - name: Checkout Code + uses: actions/checkout@{{% param "checkout_action_version" %}} + with: + fetch-depth: 0 + + # 2. Set up Docker Buildx + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} + + # 3. Cache Docker layers + - name: Cache Docker Layers + uses: actions/cache@{{% param "cache_action_version" %}} + with: + path: /tmp/.buildx-cache + key: ${{ runner.os }}-buildx-${{ github.sha }} + restore-keys: | + ${{ runner.os }}-buildx- + + # 4. Cache npm dependencies + - name: Cache npm Dependencies + uses: actions/cache@{{% param "cache_action_version" %}} + with: + path: ~/.npm + key: ${{ runner.os }}-npm-${{ hashFiles('**/package-lock.json') }} + restore-keys: | + ${{ runner.os }}-npm- + + # 5. Generate build metadata + - name: Generate Build Metadata + id: meta + run: | + echo "REPO_NAME=${GITHUB_REPOSITORY##*/}" >> "$GITHUB_OUTPUT" + echo "SHORT_SHA=${GITHUB_SHA::7}" >> "$GITHUB_OUTPUT" + + # 6. Build Docker image for testing + - name: Build Dev Docker Image + uses: docker/build-push-action@{{% param "build_push_action_version" %}} + with: + context: . + file: Dockerfile.dev + tags: ${{ steps.meta.outputs.REPO_NAME }}-dev:latest + load: true + cache-from: type=local,src=/tmp/.buildx-cache + cache-to: type=local,dest=/tmp/.buildx-cache,mode=max + + # 7. Run unit tests inside container + - name: Run Vue.js Tests + run: | + docker run --rm \ + --workdir /app \ + --entrypoint "" \ + ${{ steps.meta.outputs.REPO_NAME }}-dev:latest \ + sh -c "npm ci && npm run test -- --ci --runInBand" + env: + CI: true + NODE_ENV: test + timeout-minutes: 10 + + # 8. Log in to Docker Hub + - name: Docker Hub Login + uses: docker/login-action@{{% param "login_action_version" %}} + with: + username: ${{ secrets.DOCKER_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + + # 9. Build and push production image + - name: Build and Push Production Image + uses: docker/build-push-action@{{% param "build_push_action_version" %}} + with: + context: . + file: Dockerfile + push: true + platforms: linux/amd64,linux/arm64 + tags: | + ${{ secrets.DOCKER_USERNAME }}/${{ secrets.DOCKERHUB_PROJECT_NAME }}:latest + ${{ secrets.DOCKER_USERNAME }}/${{ secrets.DOCKERHUB_PROJECT_NAME }}:${{ steps.meta.outputs.SHORT_SHA }} + cache-from: type=local,src=/tmp/.buildx-cache +``` + +This workflow performs the following tasks for your Vue.js application: +- Triggers on every `push` or `pull request` targeting the `main` branch. +- Builds a development Docker image using `Dockerfile.dev`, optimized for testing. +- Executes unit tests using Vitest inside a clean, containerized environment to ensure consistency. +- Halts the workflow immediately if any test fails — enforcing code quality. +- Caches both Docker build layers and npm dependencies for faster CI runs. +- Authenticates securely with Docker Hub using GitHub repository secrets. +- Builds a production-ready image using the `prod` stage in `Dockerfile`. +- Tags and pushes the final image to Docker Hub with both `latest` and short SHA tags for traceability. + +> [!NOTE] +> For more information about `docker/build-push-action`, refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). + +--- + +#### Step 3: Run the workflow + +After you've added your workflow file, it's time to trigger and observe the CI/CD process in action. + +1. Commit and push your workflow file + - Select "Commit changes…" in the GitHub editor. + - This push will automatically trigger the GitHub Actions pipeline. + +2. Monitor the workflow execution + - Go to the Actions tab in your GitHub repository. + - Click into the workflow run to follow each step: **build**, **test**, and (if successful) **push**. + +3. Verify the Docker image on Docker Hub + + - After a successful workflow run, visit your [Docker Hub repositories](https://hub.docker.com/repositories). + - You should see a new image under your repository with: + - Repository name: `${your-repository-name}` + - Tags include: + - `latest` – represents the most recent successful build; ideal for quick testing or deployment. + - `` – a unique identifier based on the commit hash, useful for version tracking, rollbacks, and traceability. + +> [!TIP] Protect your main branch +> To maintain code quality and prevent accidental direct pushes, enable branch protection rules: +> - Navigate to your **GitHub repo → Settings → Branches**. +> - Under Branch protection rules, click **Add rule**. +> - Specify `main` as the branch name. +> - Enable options like: +> - *Require a pull request before merging*. +> - *Require status checks to pass before merging*. +> +> This ensures that only tested and reviewed code is merged into `main` branch. +--- + +### Summary + +In this section, you set up a complete CI/CD pipeline for your containerized Vue.js application using GitHub Actions. + +Here's what you accomplished: + +- Created a new GitHub repository specifically for your project. +- Generated a secure Docker Hub access token and added it to GitHub as a secret. +- Defined a GitHub Actions workflow that: + - Build your application inside a Docker container. + - Run tests in a consistent, containerized environment. + - Push a production-ready image to Docker Hub if tests pass. +- Triggered and verified the workflow execution through GitHub Actions. +- Confirmed that your image was successfully published to Docker Hub. + +With this setup, your Vue.js application is now ready for automated testing and deployment across environments — increasing confidence, consistency, and team productivity. + +--- + +### Related resources + +Deepen your understanding of automation and best practices for containerized apps: + +- [Introduction to GitHub Actions](/guides/gha.md) – Learn how GitHub Actions automate your workflows +- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) – Set up container builds with GitHub Actions +- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) – Full reference for writing GitHub workflows +- [Compose file reference](/compose/compose-file/) – Full configuration reference for `compose.yaml` +- [Best practices for writing Dockerfiles](/develop/develop-images/dockerfile_best-practices/) – Optimize your image for performance and security + +--- + +### Next steps + +Next, learn how you can locally test and debug your Vue.js workloads on Kubernetes before deploying. This helps you ensure your application behaves as expected in a production-like environment, reducing surprises during deployment. diff --git a/content/guides/vuejs/configure-github-actions.md b/content/guides/vuejs/configure-github-actions.md deleted file mode 100644 index 1fb6704f0784..000000000000 --- a/content/guides/vuejs/configure-github-actions.md +++ /dev/null @@ -1,320 +0,0 @@ ---- -title: Automate your builds with GitHub Actions -linkTitle: GitHub Actions CI -weight: 60 -keywords: CI/CD, GitHub( Actions), Vue.js -description: Learn how to configure CI/CD using GitHub Actions for your Vue.js application. - ---- - -## Prerequisites - -Complete all the previous sections of this guide, starting with [Containerize an Vue.js application](containerize.md). - -You must also have: -- A [GitHub](https://github.com/signup) account. -- A verified [Docker Hub](https://hub.docker.com/signup) account. - ---- - -## Overview - -In this section, you'll set up a CI/CD pipeline using [GitHub Actions](https://docs.github.com/en/actions) to automatically: - -- Build your Vue.js application inside a Docker container. -- Run tests in a consistent environment. -- Push the production-ready image to [Docker Hub](https://hub.docker.com). - ---- - -## Connect your GitHub repository to Docker Hub - -To enable GitHub Actions to build and push Docker images, you’ll securely store your Docker Hub credentials in your new GitHub repository. - -### Step 1: Generate Docker Hub credentials and set GitHub secrets - -1. Create a Personal Access Token (PAT) from [Docker Hub](https://hub.docker.com) - 1. Go to your **Docker Hub account → Account Settings → Security**. - 2. Generate a new Access Token with **Read/Write** permissions. - 3. Name it something like `docker-vuejs-sample`. - 4. Copy and save the token — you’ll need it in Step 4. - -2. Create a repository in [Docker Hub](https://hub.docker.com/repositories/) - 1. Go to your **Docker Hub account → Create a repository**. - 2. For the Repository Name, use something descriptive — for example: `vuejs-sample`. - 3. Once created, copy and save the repository name — you’ll need it in Step 4. - -3. Create a new [GitHub repository](https://github.com/new) for your Vue.js project - -4. Add Docker Hub credentials as GitHub repository secrets - - In your newly created GitHub repository: - - 1. Navigate to: - **Settings → Secrets and variables → Actions → New repository secret**. - - 2. Add the following secrets: - - | Name | Value | - |-------------------|--------------------------------| - | `DOCKER_USERNAME` | Your Docker Hub username | - | `DOCKERHUB_TOKEN` | Your Docker Hub access token (created in Step 1) | - | `DOCKERHUB_PROJECT_NAME` | Your Docker Project Name (created in Step 2) | - - These secrets allow GitHub Actions to authenticate securely with Docker Hub during automated workflows. - -5. Connect Your Local Project to GitHub - - Link your local project `docker-vuejs-sample` to the GitHub repository you just created by running the following command from your project root: - - ```console - $ git remote set-url origin https://github.com/{your-username}/{your-repository-name}.git - ``` - - >[!IMPORTANT] - >Replace `{your-username}` and `{your-repository}` with your actual GitHub username and repository name. - - To confirm that your local project is correctly connected to the remote GitHub repository, run: - - ```console - $ git remote -v - ``` - - You should see output similar to: - - ```console - origin https://github.com/{your-username}/{your-repository-name}.git (fetch) - origin https://github.com/{your-username}/{your-repository-name}.git (push) - ``` - - This confirms that your local repository is properly linked and ready to push your source code to GitHub. - -6. Push your source code to GitHub - - Follow these steps to commit and push your local project to your GitHub repository: - - 1. Stage all files for commit. - - ```console - $ git add -A - ``` - This command stages all changes — including new, modified, and deleted files — preparing them for commit. - - - 2. Commit the staged changes with a descriptive message. - - ```console - $ git commit -m "Initial commit" - ``` - This command creates a commit that snapshots the staged changes with a descriptive message. - - 3. Push the code to the `main` branch. - - ```console - $ git push -u origin main - ``` - This command pushes your local commits to the `main` branch of the remote GitHub repository and sets the upstream branch. - -Once completed, your code will be available on GitHub, and any GitHub Actions workflow you’ve configured will run automatically. - -> [!NOTE] -> Learn more about the Git commands used in this step: -> - [Git add](https://git-scm.com/docs/git-add) – Stage changes (new, modified, deleted) for commit -> - [Git commit](https://git-scm.com/docs/git-commit) – Save a snapshot of your staged changes -> - [Git push](https://git-scm.com/docs/git-push) – Upload local commits to your GitHub repository -> - [Git remote](https://git-scm.com/docs/git-remote) – View and manage remote repository URLs - ---- - -### Step 2: Set up the workflow - -Now you'll create a GitHub Actions workflow that builds your Docker image, runs tests, and pushes the image to Docker Hub. - -1. Go to your repository on GitHub and select the **Actions** tab in the top menu. - -2. Select **Set up a workflow yourself** when prompted. - - This opens an inline editor to create a new workflow file. By default, it will be saved to: - `.github/workflows/main.yml` - - -3. Add the following workflow configuration to the new file: - -```yaml -name: CI/CD – Vue.js App with Docker - -on: - push: - branches: [main] - pull_request: - branches: [main] - types: [opened, synchronize, reopened] - -jobs: - build-test-deploy: - name: Build, Test & Deploy - runs-on: ubuntu-latest - - steps: - # 1. Checkout the codebase - - name: Checkout Code - uses: actions/checkout@{{% param "checkout_action_version" %}} - with: - fetch-depth: 0 - - # 2. Set up Docker Buildx - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} - - # 3. Cache Docker layers - - name: Cache Docker Layers - uses: actions/cache@{{% param "cache_action_version" %}} - with: - path: /tmp/.buildx-cache - key: ${{ runner.os }}-buildx-${{ github.sha }} - restore-keys: | - ${{ runner.os }}-buildx- - - # 4. Cache npm dependencies - - name: Cache npm Dependencies - uses: actions/cache@{{% param "cache_action_version" %}} - with: - path: ~/.npm - key: ${{ runner.os }}-npm-${{ hashFiles('**/package-lock.json') }} - restore-keys: | - ${{ runner.os }}-npm- - - # 5. Generate build metadata - - name: Generate Build Metadata - id: meta - run: | - echo "REPO_NAME=${GITHUB_REPOSITORY##*/}" >> "$GITHUB_OUTPUT" - echo "SHORT_SHA=${GITHUB_SHA::7}" >> "$GITHUB_OUTPUT" - - # 6. Build Docker image for testing - - name: Build Dev Docker Image - uses: docker/build-push-action@{{% param "build_push_action_version" %}} - with: - context: . - file: Dockerfile.dev - tags: ${{ steps.meta.outputs.REPO_NAME }}-dev:latest - load: true - cache-from: type=local,src=/tmp/.buildx-cache - cache-to: type=local,dest=/tmp/.buildx-cache,mode=max - - # 7. Run unit tests inside container - - name: Run Vue.js Tests - run: | - docker run --rm \ - --workdir /app \ - --entrypoint "" \ - ${{ steps.meta.outputs.REPO_NAME }}-dev:latest \ - sh -c "npm ci && npm run test -- --ci --runInBand" - env: - CI: true - NODE_ENV: test - timeout-minutes: 10 - - # 8. Log in to Docker Hub - - name: Docker Hub Login - uses: docker/login-action@{{% param "login_action_version" %}} - with: - username: ${{ secrets.DOCKER_USERNAME }} - password: ${{ secrets.DOCKERHUB_TOKEN }} - - # 9. Build and push production image - - name: Build and Push Production Image - uses: docker/build-push-action@{{% param "build_push_action_version" %}} - with: - context: . - file: Dockerfile - push: true - platforms: linux/amd64,linux/arm64 - tags: | - ${{ secrets.DOCKER_USERNAME }}/${{ secrets.DOCKERHUB_PROJECT_NAME }}:latest - ${{ secrets.DOCKER_USERNAME }}/${{ secrets.DOCKERHUB_PROJECT_NAME }}:${{ steps.meta.outputs.SHORT_SHA }} - cache-from: type=local,src=/tmp/.buildx-cache -``` - -This workflow performs the following tasks for your Vue.js application: -- Triggers on every `push` or `pull request` targeting the `main` branch. -- Builds a development Docker image using `Dockerfile.dev`, optimized for testing. -- Executes unit tests using Vitest inside a clean, containerized environment to ensure consistency. -- Halts the workflow immediately if any test fails — enforcing code quality. -- Caches both Docker build layers and npm dependencies for faster CI runs. -- Authenticates securely with Docker Hub using GitHub repository secrets. -- Builds a production-ready image using the `prod` stage in `Dockerfile`. -- Tags and pushes the final image to Docker Hub with both `latest` and short SHA tags for traceability. - -> [!NOTE] -> For more information about `docker/build-push-action`, refer to the [GitHub Action README](https://github.com/docker/build-push-action/blob/master/README.md). - ---- - -### Step 3: Run the workflow - -After you've added your workflow file, it's time to trigger and observe the CI/CD process in action. - -1. Commit and push your workflow file - - Select "Commit changes…" in the GitHub editor. - - This push will automatically trigger the GitHub Actions pipeline. - -2. Monitor the workflow execution - - Go to the Actions tab in your GitHub repository. - - Click into the workflow run to follow each step: **build**, **test**, and (if successful) **push**. - -3. Verify the Docker image on Docker Hub - - - After a successful workflow run, visit your [Docker Hub repositories](https://hub.docker.com/repositories). - - You should see a new image under your repository with: - - Repository name: `${your-repository-name}` - - Tags include: - - `latest` – represents the most recent successful build; ideal for quick testing or deployment. - - `` – a unique identifier based on the commit hash, useful for version tracking, rollbacks, and traceability. - -> [!TIP] Protect your main branch -> To maintain code quality and prevent accidental direct pushes, enable branch protection rules: -> - Navigate to your **GitHub repo → Settings → Branches**. -> - Under Branch protection rules, click **Add rule**. -> - Specify `main` as the branch name. -> - Enable options like: -> - *Require a pull request before merging*. -> - *Require status checks to pass before merging*. -> -> This ensures that only tested and reviewed code is merged into `main` branch. ---- - -## Summary - -In this section, you set up a complete CI/CD pipeline for your containerized Vue.js application using GitHub Actions. - -Here's what you accomplished: - -- Created a new GitHub repository specifically for your project. -- Generated a secure Docker Hub access token and added it to GitHub as a secret. -- Defined a GitHub Actions workflow that: - - Build your application inside a Docker container. - - Run tests in a consistent, containerized environment. - - Push a production-ready image to Docker Hub if tests pass. -- Triggered and verified the workflow execution through GitHub Actions. -- Confirmed that your image was successfully published to Docker Hub. - -With this setup, your Vue.js application is now ready for automated testing and deployment across environments — increasing confidence, consistency, and team productivity. - ---- - -## Related resources - -Deepen your understanding of automation and best practices for containerized apps: - -- [Introduction to GitHub Actions](/guides/gha.md) – Learn how GitHub Actions automate your workflows -- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) – Set up container builds with GitHub Actions -- [Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) – Full reference for writing GitHub workflows -- [Compose file reference](/compose/compose-file/) – Full configuration reference for `compose.yaml` -- [Best practices for writing Dockerfiles](/develop/develop-images/dockerfile_best-practices/) – Optimize your image for performance and security - ---- - -## Next steps - -Next, learn how you can locally test and debug your Vue.js workloads on Kubernetes before deploying. This helps you ensure your application behaves as expected in a production-like environment, reducing surprises during deployment. diff --git a/content/guides/vuejs/containerize.md b/content/guides/vuejs/containerize.md deleted file mode 100644 index c02c217ae134..000000000000 --- a/content/guides/vuejs/containerize.md +++ /dev/null @@ -1,530 +0,0 @@ ---- -title: Containerize an Vue.js Application -linkTitle: Containerize -weight: 10 -keywords: vue.js, vue, js, node, image, initialize, build -description: Learn how to containerize an Vue.js application with Docker by creating an optimized, production-ready image using best practices for performance, security, and scalability. - ---- - - -## Prerequisites - -Before you begin, make sure the following tools are installed and available on your system: - -- You have installed the latest version of [Docker Desktop](/get-started/get-docker.md). -- You have a [git client](https://git-scm.com/downloads). The examples in this section use a command-line based git client, but you can use any client. - -> **New to Docker?** -> Start with the [Docker basics](/get-started/docker-concepts/the-basics/what-is-a-container.md) guide to get familiar with key concepts like images, containers, and Dockerfiles. - ---- - -## Overview - -This guide walks you through the complete process of containerizing an Vue.js application with Docker. You’ll learn how to create a production-ready Docker image using best practices that improve performance, security, scalability, and deployment efficiency. - -By the end of this guide, you will: - -- Containerize an Vue.js application using Docker. -- Create and optimize a Dockerfile for production builds. -- Use multi-stage builds to minimize image size. -- Serve the application efficiently with a custom Nginx configuration. -- Build secure and maintainable Docker images by following best practices. - ---- - -## Get the sample application - -Clone the sample application to use with this guide. Open a terminal, navigate to the directory where you want to work, and run the following command -to clone the git repository: - -```console -$ git clone https://github.com/kristiyan-velkov/docker-vuejs-sample -``` ---- - -## Build the Docker image - -Vue.js is a front-end framework that compiles into static assets, so the Dockerfile is customized to align with how Vue.js applications are built and efficiently served in a production environment. - -> [!TIP] -> -> [Gordon](/ai/gordon/), Docker's AI assistant, can generate Docker assets for your project. Ask Gordon to create a Dockerfile, Compose file, and `.dockerignore` tailored to your application. - -### Step 1: Create the Dockerfile - -Before creating a Dockerfile, you need to choose a base image. You can either use the [Node.js Official Image](https://hub.docker.com/_/node) or a Docker Hardened Image (DHI) from the [Hardened Image catalog](https://hub.docker.com/hardened-images/catalog). - -Choosing DHI offers the advantage of a production-ready image that is lightweight and secure. For more information, see [Docker Hardened Images](https://docs.docker.com/dhi/). - -> [!IMPORTANT] -> This guide uses a stable Node.js LTS image tag that is considered secure when the guide is written. Because new releases and security patches are published regularly, the tag shown here may no longer be the safest option when you follow the guide. Always review the latest available image tags and select a secure, up-to-date version before building or deploying your application. -> -> Official Node.js Docker Images: https://hub.docker.com/_/node - -{{< tabs >}} -{{< tab name="Using Docker Hardened Images" >}} -Docker Hardened Images (DHIs) are available for Node.js in the [Docker Hardened Images catalog](https://hub.docker.com/hardened-images/catalog/dhi/node). Docker Hardened Images are freely available to everyone with no subscription required. You can pull and use them like any other Docker image after signing in to the DHI registry. For more information, see the [DHI quickstart](/dhi/get-started/) guide. - -1. Sign in to the DHI registry: - ```console - $ docker login dhi.io - ``` - -2. Pull the Node.js DHI (check the catalog for available versions): - ```console - $ docker pull dhi.io/node:24-alpine3.22-dev - ``` - -3. Pull the Nginx DHI (check the catalog for available versions): - ```console - $ docker pull dhi.io/nginx:1.28.0-alpine3.21-dev - ``` - -In the following Dockerfile, the `FROM` instructions use `dhi.io/node:24-alpine3.22-dev` and `dhi.io/nginx:1.28.0-alpine3.21-dev` as the base images. - -```dockerfile -# ========================================= -# Stage 1: Build the Vue.js Application -# ========================================= -# Use a lightweight DHI Node.js image for building -FROM dhi.io/node:24-alpine3.22-dev AS builder - -# Set the working directory inside the container -WORKDIR /app - -# Copy package-related files first to leverage Docker's caching mechanism -COPY package.json package-lock.json* ./ - -# Install project dependencies using npm ci (ensures a clean, reproducible install) -RUN --mount=type=cache,target=/root/.npm npm ci - -# Copy the rest of the application source code into the container -COPY . . - -# Build the Vue.js application -RUN npm run build - -# ========================================= -# Stage 2: Prepare Nginx to Serve Static Files -# ========================================= - -FROM dhi.io/nginx:1.28.0-alpine3.21-dev AS runner - -# Copy custom Nginx config -COPY nginx.conf /etc/nginx/nginx.conf - -# Copy the static build output from the build stage to Nginx's default HTML serving directory -COPY --chown=nginx:nginx --from=builder /app/dist /usr/share/nginx/html - -# Use a built-in non-root user for security best practices -USER nginx - -# Expose port 8080 to allow HTTP traffic -# Note: The default Nginx container now listens on port 8080 instead of 80 -EXPOSE 8080 - -# Start Nginx directly with custom config -ENTRYPOINT ["nginx", "-c", "/etc/nginx/nginx.conf"] -CMD ["-g", "daemon off;"] -``` - -{{< /tab >}} -{{< tab name="Using the Docker Official Image" >}} - -Create a file named `Dockerfile` with the following contents: - -```dockerfile -# ========================================= -# Stage 1: Build the Vue.js Application -# ========================================= -ARG NODE_VERSION=24.12.0-alpine -ARG NGINX_VERSION=alpine3.22 - -# Use a lightweight Node.js image for building (customizable via ARG) -FROM node:${NODE_VERSION} AS builder - -# Set the working directory inside the container -WORKDIR /app - -# Copy package-related files first to leverage Docker's caching mechanism -COPY package.json package-lock.json* ./ - -# Install project dependencies using npm ci (ensures a clean, reproducible install) -RUN --mount=type=cache,target=/root/.npm npm ci - -# Copy the rest of the application source code into the container -COPY . . - -# Build the Vue.js application -RUN npm run build - -# ========================================= -# Stage 2: Prepare Nginx to Serve Static Files -# ========================================= - -FROM nginxinc/nginx-unprivileged:${NGINX_VERSION} AS runner - -# Copy custom Nginx config -COPY nginx.conf /etc/nginx/nginx.conf - -# Copy the static build output from the build stage to Nginx's default HTML serving directory -COPY --chown=nginx:nginx --from=builder /app/dist /usr/share/nginx/html - -# Use a built-in non-root user for security best practices -USER nginx - -# Expose port 8080 to allow HTTP traffic -# Note: The default Nginx container now listens on port 8080 instead of 80 -EXPOSE 8080 - -# Start Nginx directly with custom config -ENTRYPOINT ["nginx", "-c", "/etc/nginx/nginx.conf"] -CMD ["-g", "daemon off;"] -``` - -> [!NOTE] -> We are using nginx-unprivileged instead of the standard Nginx image to follow security best practices. -> Running as a non-root user in the final image: ->- Reduces the attack surface ->- Aligns with Docker’s recommendations for container hardening ->- Helps comply with stricter security policies in production environments - -{{< /tab >}} -{{< /tabs >}} - -### Step 2: Create the compose.yaml file - -Create a file named `compose.yaml` with the following contents: - -```yaml {collapse=true,title=compose.yaml} -services: - server: - build: - context: . - ports: - - 8080:8080 -``` - -### Step 3: Create the .dockerignore file - -The `.dockerignore` file plays a crucial role in optimizing your Docker image by specifying which files and directories should be excluded from the build context. - -> [!NOTE] ->This helps: ->- Reduce image size ->- Speed up the build process ->- Prevent sensitive or unnecessary files (like `.env`, `.git`, or `node_modules`) from being added to the final image. -> -> To learn more, visit the [.dockerignore reference](/reference/dockerfile.md#dockerignore-file). - -Create a file named `.dockerignore` with the following contents: - -```dockerignore -# ------------------------------- -# Dependency directories -# ------------------------------- -node_modules/ - -# ------------------------------- -# Production and build outputs -# ------------------------------- -dist/ -out/ -build/ -public/build/ - -# ------------------------------- -# Vite, VuePress, and cache dirs -# ------------------------------- -.vite/ -.vitepress/ -.cache/ -.tmp/ - -# ------------------------------- -# Test output and coverage -# ------------------------------- -coverage/ -reports/ -jest/ -cypress/ -cypress/screenshots/ -cypress/videos/ - -# ------------------------------- -# Environment and config files -# ------------------------------- -*.env* -!.env.production # Keep production env if needed -*.local -*.log - -# ------------------------------- -# TypeScript artifacts -# ------------------------------- -*.tsbuildinfo - -# ------------------------------- -# Editor and IDE config -# ------------------------------- -.vscode/ -.idea/ -*.swp - -# ------------------------------- -# System files -# ------------------------------- -.DS_Store -Thumbs.db - -# ------------------------------- -# Lockfiles (optional) -# ------------------------------- -npm-debug.log* -yarn-debug.log* -yarn-error.log* -pnpm-debug.log* - -# ------------------------------- -# Git files -# ------------------------------- -.git/ -.gitignore - -# ------------------------------- -# Docker-related files -# ------------------------------- -Dockerfile -.dockerignore -docker-compose.yml -docker-compose.override.yml -``` - -### Step 4: Create the `nginx.conf` file - -To serve your Vue.js application efficiently inside the container, you’ll configure Nginx with a custom setup. This configuration is optimized for performance, browser caching, gzip compression, and support for client-side routing. - -Create a file named `nginx.conf` in the root of your project directory, and add the following content: - -> [!NOTE] -> To learn more about configuring Nginx, see the [official Nginx documentation](https://nginx.org/en/docs/). - -```nginx -worker_processes auto; -pid /tmp/nginx.pid; - -events { - worker_connections 1024; -} - -http { - include /etc/nginx/mime.types; - default_type application/octet-stream; - charset utf-8; - - access_log off; - error_log /dev/stderr warn; - - sendfile on; - tcp_nopush on; - tcp_nodelay on; - keepalive_timeout 65; - keepalive_requests 1000; - - gzip on; - gzip_comp_level 6; - gzip_proxied any; - gzip_min_length 256; - gzip_vary on; - gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript image/svg+xml; - - server { - listen 8080; - server_name localhost; - - root /usr/share/nginx/html; - index index.html; - - location / { - try_files $uri $uri/ /index.html; - } - - location ~* \.(?:ico|css|js|gif|jpe?g|png|woff2?|eot|ttf|svg|map)$ { - expires 1y; - access_log off; - add_header Cache-Control "public, immutable"; - add_header X-Content-Type-Options nosniff; - } - - location /assets/ { - expires 1y; - add_header Cache-Control "public, immutable"; - add_header X-Content-Type-Options nosniff; - } - - error_page 404 /index.html; - } -} -``` - -### Step 5: Build the Vue.js application image - -With your custom configuration in place, you're now ready to build the Docker image for your Vue.js application. - -The updated setup includes: - -- The updated setup includes a clean, production-ready Nginx configuration tailored specifically for Vue.js. -- Efficient multi-stage Docker build, ensuring a small and secure final image. - -After completing the previous steps, your project directory should now contain the following files: - -```text -├── docker-vuejs-sample/ -│ ├── Dockerfile -│ ├── .dockerignore -│ ├── compose.yaml -│ └── nginx.conf -``` - -Now that your Dockerfile is configured, you can build the Docker image for your Vue.js application. - -> [!NOTE] -> The `docker build` command packages your application into an image using the instructions in the Dockerfile. It includes all necessary files from the current directory (called the [build context](/build/concepts/context/#what-is-a-build-context)). - -Run the following command from the root of your project: - -```console -$ docker build --tag docker-vuejs-sample . -``` - -What this command does: -- Uses the Dockerfile in the current directory (.) -- Packages the application and its dependencies into a Docker image -- Tags the image as docker-vuejs-sample so you can reference it later - - -### Step 6: View local images - -After building your Docker image, you can check which images are available on your local machine using either the Docker CLI or [Docker Desktop](/manuals/desktop/use-desktop/images.md). Since you're already working in the terminal, let's use the Docker CLI. - -To list all locally available Docker images, run the following command: - -```console -$ docker images -``` - -Example Output: - -```shell -REPOSITORY TAG IMAGE ID CREATED SIZE -docker-vuejs-sample latest 8c9c199179d4 14 seconds ago 76.2MB -``` - -This output provides key details about your images: - -- **Repository** – The name assigned to the image. -- **Tag** – A version label that helps identify different builds (e.g., latest). -- **Image ID** – A unique identifier for the image. -- **Created** – The timestamp indicating when the image was built. -- **Size** – The total disk space used by the image. - -If the build was successful, you should see `docker-vuejs-sample` image listed. - ---- - -## Run the containerized application - -In the previous step, you created a Dockerfile for your Vue.js application and built a Docker image using the docker build command. Now it’s time to run that image in a container and verify that your application works as expected. - - -Inside the `docker-vuejs-sample` directory, run the following command in a -terminal. - -```console -$ docker compose up --build -``` - -Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see a simple Vue.js web application. - -Press `ctrl+c` in the terminal to stop your application. - -### Run the application in the background - -You can run the application detached from the terminal by adding the `-d` -option. Inside the `docker-vuejs-sample` directory, run the following command -in a terminal. - -```console -$ docker compose up --build -d -``` - -Open a browser and view the application at [http://localhost:8080](http://localhost:8080). You should see your Vue.js application running in the browser. - - -To confirm that the container is running, use `docker ps` command: - -```console -$ docker ps -``` - -This will list all active containers along with their ports, names, and status. Look for a container exposing port 8080. - -Example Output: - -```shell -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -37a1fa85e4b0 docker-vuejs-sample-server "nginx -c /etc/nginx…" About a minute ago Up About a minute 0.0.0.0:8080->8080/tcp docker-vuejs-sample-server-1 -``` - - -To stop the application, run: - -```console -$ docker compose down -``` - - -> [!NOTE] -> For more information about Compose commands, see the [Compose CLI -> reference](/reference/cli/docker/compose/). - ---- - -## Summary - -In this guide, you learned how to containerize, build, and run an Vue.js application using Docker. By following best practices, you created a secure, optimized, and production-ready setup. - -What you accomplished: -- Created a multi-stage `Dockerfile` that compiles the Vue.js application and serves the static files using Nginx. -- Created a `.dockerignore` file to exclude unnecessary files and keep the image clean and efficient. -- Built your Docker image using `docker build`. -- Ran the container using `docker compose up`, both in the foreground and in detached mode. -- Verified that the app was running by visiting [http://localhost:8080](http://localhost:8080). -- Learned how to stop the containerized application using `docker compose down`. - -You now have a fully containerized Vue.js application, running in a Docker container, and ready for deployment across any environment with confidence and consistency. - ---- - -## Related resources - -Explore official references and best practices to sharpen your Docker workflow: - -- [Multi-stage builds](/build/building/multi-stage/) – Learn how to separate build and runtime stages. -- [Best practices for writing Dockerfiles](/develop/develop-images/dockerfile_best-practices/) – Write efficient, maintainable, and secure Dockerfiles. -- [Build context in Docker](/build/concepts/context/) – Learn how context affects image builds. -- [`docker build` CLI reference](/reference/cli/docker/image/build/) – Build Docker images from a Dockerfile. -- [`docker images` CLI reference](/reference/cli/docker/image/ls/) – Manage and inspect local Docker images. -- [`docker compose up` CLI reference](/reference/cli/docker/compose/up/) – Start and run multi-container applications. -- [`docker compose down` CLI reference](/reference/cli/docker/compose/down/) – Stop and remove containers, networks, and volumes. - ---- - -## Next steps - -With your Vue.js application now containerized, you're ready to move on to the next step. - -In the next section, you'll learn how to develop your application using Docker containers, enabling a consistent, isolated, and reproducible development environment across any machine. - diff --git a/content/guides/vuejs/deploy.md b/content/guides/vuejs/deploy.md deleted file mode 100644 index 678f8956e055..000000000000 --- a/content/guides/vuejs/deploy.md +++ /dev/null @@ -1,201 +0,0 @@ ---- -title: Test your Vue.js deployment -linkTitle: Test your deployment -weight: 60 -keywords: deploy, kubernetes, vue, vue.js -description: Learn how to deploy locally to test and debug your Kubernetes deployment - ---- - -## Prerequisites - -Before you begin, make sure you’ve completed the following: -- Complete all the previous sections of this guide, starting with [Containerize Vue.js application](containerize.md). -- [Enable Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. - -> **New to Kubernetes?** -> Visit the [Kubernetes basics tutorial](https://kubernetes.io/docs/tutorials/kubernetes-basics/) to get familiar with how clusters, pods, deployments, and services work. - ---- - -## Overview - -This section guides you through deploying your containerized Vue.js application locally using [Docker Desktop’s built-in Kubernetes](/desktop/kubernetes/). Running your app in a local Kubernetes cluster closely simulates a real production environment, enabling you to test, validate, and debug your workloads with confidence before promoting them to staging or production. - ---- - -## Create a Kubernetes YAML file - -Follow these steps to define your deployment configuration: - -1. In the root of your project, create a new file named: vuejs-sample-kubernetes.yaml - -2. Open the file in your IDE or preferred text editor. - -3. Add the following configuration, and be sure to replace `{DOCKER_USERNAME}` and `{DOCKERHUB_PROJECT_NAME}` with your actual Docker Hub username and repository name from the previous [Automate your builds with GitHub Actions](configure-github-actions.md). - - -```yaml -apiVersion: apps/v1 -kind: Deployment -metadata: - name: vuejs-sample - namespace: default -spec: - replicas: 1 - selector: - matchLabels: - app: vuejs-sample - template: - metadata: - labels: - app: vuejs-sample - spec: - containers: - - name: vuejs-container - image: {DOCKER_USERNAME}/{DOCKERHUB_PROJECT_NAME}:latest - imagePullPolicy: Always - ports: - - containerPort: 8080 - resources: - limits: - cpu: "500m" - memory: "256Mi" - requests: - cpu: "250m" - memory: "128Mi" ---- -apiVersion: v1 -kind: Service -metadata: - name: vuejs-sample-service - namespace: default -spec: - type: NodePort - selector: - app: vuejs-sample - ports: - - port: 8080 - targetPort: 8080 - nodePort: 30001 -``` - -This manifest defines two key Kubernetes resources, separated by `---`: - -- Deployment - Deploys a single replica of your Vue.js application inside a pod. The pod uses the Docker image built and pushed by your GitHub Actions CI/CD workflow - (refer to [Automate your builds with GitHub Actions](configure-github-actions.md)). - The container listens on port `8080`, which is typically used by [Nginx](https://nginx.org/en/docs/) to serve your production Vue.js app. - -- Service (NodePort) - Exposes the deployed pod to your local machine. - It forwards traffic from port `30001` on your host to port `8080` inside the container. - This lets you access the application in your browser at [http://localhost:30001](http://localhost:30001). - -> [!NOTE] -> To learn more about Kubernetes objects, see the [Kubernetes documentation](https://kubernetes.io/docs/home/). - ---- - -## Deploy and check your application - -Follow these steps to deploy your containerized Vue.js app into a local Kubernetes cluster and verify that it’s running correctly. - -### Step 1. Apply the Kubernetes configuration - -In your terminal, navigate to the directory where your `vuejs-sample-kubernetes.yaml` file is located, then deploy the resources using: - -```console - $ kubectl apply -f vuejs-sample-kubernetes.yaml -``` - -If everything is configured properly, you’ll see confirmation that both the Deployment and the Service were created: - -```shell - deployment.apps/vuejs-sample created - service/vuejs-sample-service created -``` - -This confirms that both the Deployment and the Service were successfully created and are now running inside your local cluster. - -### Step 2. Check the deployment status - -Run the following command to check the status of your deployment: - -```console - $ kubectl get deployments -``` - -You should see output similar to the following: - -```shell - NAME READY UP-TO-DATE AVAILABLE AGE - vuejs-sample 1/1 1 1 1m14s -``` - -This confirms that your pod is up and running with one replica available. - -### Step 3. Verify the service exposure - -Check if the NodePort service is exposing your app to your local machine: - -```console -$ kubectl get services -``` - -You should see something like: - -```shell -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -vuejs-sample-service NodePort 10.98.233.59 8080:30001/TCP 1m -``` - -This output confirms that your app is available via NodePort on port 30001. - -### Step 4. Access your app in the browser - -Open your browser and navigate to [http://localhost:30001](http://localhost:30001). - -You should see your production-ready Vue.js Sample application running — served by your local Kubernetes cluster. - -### Step 5. Clean up Kubernetes resources - -Once you're done testing, you can delete the deployment and service using: - -```console - $ kubectl delete -f vuejs-sample-kubernetes.yaml -``` - -Expected output: - -```shell - deployment.apps "vuejs-sample" deleted - service "vuejs-sample-service" deleted -``` - -This ensures your cluster stays clean and ready for the next deployment. - ---- - -## Summary - -In this section, you learned how to deploy your Vue.js application to a local Kubernetes cluster using Docker Desktop. This setup allows you to test and debug your containerized app in a production-like environment before deploying it to the cloud. - -What you accomplished: - -- Created a Kubernetes Deployment and NodePort Service for your Vue.js app -- Used `kubectl apply` to deploy the application locally -- Verified the app was running and accessible at `http://localhost:30001` -- Cleaned up your Kubernetes resources after testing - ---- - -## Related resources - -Explore official references and best practices to sharpen your Kubernetes deployment workflow: - -- [Kubernetes documentation](https://kubernetes.io/docs/home/) – Learn about core concepts, workloads, services, and more. -- [Deploy on Kubernetes with Docker Desktop](/manuals) – Use Docker Desktop’s built-in Kubernetes support for local testing and development. -- [`kubectl` CLI reference](https://kubernetes.io/docs/reference/kubectl/) – Manage Kubernetes clusters from the command line. -- [Kubernetes Deployment resource](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) – Understand how to manage and scale applications using Deployments. -- [Kubernetes Service resource](https://kubernetes.io/docs/concepts/services-networking/service/) – Learn how to expose your application to internal and external traffic. \ No newline at end of file diff --git a/content/guides/vuejs/develop.md b/content/guides/vuejs/develop.md deleted file mode 100644 index 38614f8a8693..000000000000 --- a/content/guides/vuejs/develop.md +++ /dev/null @@ -1,189 +0,0 @@ ---- -title: Use containers for Vue.js development -linkTitle: Develop your app -weight: 30 -keywords: vuejs, development, node -description: Learn how to develop your Vue.js application locally using containers. - ---- - -## Prerequisites - -Complete [Containerize Vue.js application](containerize.md). - ---- - -## Overview - -In this section, you'll set up both production and development environments for your Vue.js application using Docker Compose. This approach streamlines your workflow—delivering a lightweight, static site via Nginx in production, and providing a fast, live-reloading dev server with Compose Watch for efficient local development. - -You’ll learn how to: -- Configure isolated environments: Set up separate containers optimized for production and development use cases. -- Live-reload in development: Use Compose Watch to automatically sync file changes, enabling real-time updates without manual intervention. -- Preview and debug with ease: Develop inside containers with a seamless preview and debug experience—no rebuilds required after every change. - ---- - -## Automatically update services (development mode) - -Leverage Compose Watch to enable real-time file synchronization between your local machine and the containerized Vue.js development environment. This powerful feature eliminates the need to manually rebuild or restart containers, providing a fast, seamless, and efficient development workflow. - -With Compose Watch, your code updates are instantly reflected inside the container—perfect for rapid testing, debugging, and live previewing changes. - -## Step 1: Create a development Dockerfile - -Create a file named `Dockerfile.dev` in your project root with the following content: - -```dockerfile -# ========================================= -# Stage 1: Develop the Vue.js Application -# ========================================= -ARG NODE_VERSION=24.12.0-alpine - -# Use a lightweight Node.js image for development -FROM node:${NODE_VERSION} AS dev - -# Set environment variable to indicate development mode -ENV NODE_ENV=development - -# Set the working directory inside the container -WORKDIR /app - -# Copy package-related files first to leverage Docker's caching mechanism -COPY package.json package-lock.json* ./ - -# Install project dependencies -RUN --mount=type=cache,target=/root/.npm npm install - -# Copy the rest of the application source code into the container -COPY . . - -# Change ownership of the application directory to the node user -RUN chown -R node:node /app - -# Switch to the node user -USER node - -# Expose the port used by the Vite development server -EXPOSE 5173 - -# Use a default command, can be overridden in Docker compose.yml file -CMD [ "npm", "run", "dev", "--", "--host" ] - -``` - -This file sets up a lightweight development environment for your Vue.js application using the dev server. - -### Step 2: Update your `compose.yaml` file - -Open your `compose.yaml` file and define two services: one for production (`vuejs-prod`) and one for development (`vuejs-dev`). - -Here’s an example configuration for an Vue.js application: - -```yaml -services: - vuejs-prod: - build: - context: . - dockerfile: Dockerfile - image: docker-vuejs-sample - ports: - - "8080:8080" - - vuejs-dev: - build: - context: . - dockerfile: Dockerfile.dev - ports: - - "5173:5173" - develop: - watch: - - path: ./src - target: /app/src - action: sync - - path: ./package.json - target: /app/package.json - action: restart - - path: ./vite.config.js - target: /app/vite.config.js - action: restart -``` -- The `vuejs-prod` service builds and serves your static production app using Nginx. -- The `vuejs-dev` service runs your Vue.js development server with live reload and hot module replacement. -- `watch` triggers file sync with Compose Watch. - -> [!NOTE] -> For more details, see the official guide: [Use Compose Watch](/manuals/compose/how-tos/file-watch.md). - -After completing the previous steps, your project directory should now contain the following files: - -```text -├── docker-vuejs-sample/ -│ ├── Dockerfile -│ ├── Dockerfile.dev -│ ├── .dockerignore -│ ├── compose.yaml -│ └── nginx.conf -``` - -### Step 4: Start Compose Watch - -Run the following command from the project root to start the container in watch mode - -```console -$ docker compose watch vuejs-dev -``` - -### Step 5: Test Compose Watch with Vue.js - -To confirm that Compose Watch is functioning correctly: - -1. Open the `src/App.vue` file in your text editor. - -2. Locate the following line: - - ```html - - ``` - -3. Change it to: - - ```html - - ``` - -4. Save the file. - -5. Open your browser at [http://localhost:5173](http://localhost:5173). - -You should see the updated text appear instantly, without needing to rebuild the container manually. This confirms that file watching and automatic synchronization are working as expected. - ---- - -## Summary - -In this section, you set up a complete development and production workflow for your Vue.js application using Docker and Docker Compose. - -Here’s what you accomplished: -- Created a `Dockerfile.dev` to streamline local development with hot reloading -- Defined separate `vuejs-dev` and `vuejs-prod` services in your `compose.yaml` file -- Enabled real-time file syncing using Compose Watch for a smoother development experience -- Verified that live updates work seamlessly by modifying and previewing a component - -With this setup, you're now equipped to build, run, and iterate on your Vue.js app entirely within containers—efficiently and consistently across environments. - ---- - -## Related resources - -Deepen your knowledge and improve your containerized development workflow with these guides: - -- [Using Compose Watch](/manuals/compose/how-tos/file-watch.md) – Automatically sync source changes during development -- [Multi-stage builds](/manuals/build/building/multi-stage.md) – Create efficient, production-ready Docker images -- [Dockerfile best practices](/build/building/best-practices/) – Write clean, secure, and optimized Dockerfiles. -- [Compose file reference](/compose/compose-file/) – Learn the full syntax and options available for configuring services in `compose.yaml`. -- [Docker volumes](/storage/volumes/) – Persist and manage data between container runs - -## Next steps - -In the next section, you'll learn how to run unit tests for your Vue.js application inside Docker containers. This ensures consistent testing across all environments and removes dependencies on local machine setup. diff --git a/content/guides/vuejs/run-tests.md b/content/guides/vuejs/run-tests.md deleted file mode 100644 index 7e1f2d76897e..000000000000 --- a/content/guides/vuejs/run-tests.md +++ /dev/null @@ -1,138 +0,0 @@ ---- -title: Run vue.js tests in a container -linkTitle: Run your tests -weight: 40 -keywords: vue.js, vue, test, vitest -description: Learn how to run your vue.js tests in a container. - ---- - -## Prerequisites - -Complete all the previous sections of this guide, starting with [Containerize Vue.js application](containerize.md). - -## Overview - -Testing is a critical part of the development process. In this section, you'll learn how to: - -- Run unit tests using Vitest inside a Docker container. -- Use Docker Compose to run tests in an isolated, reproducible environment. - -You’ll use [Vitest](https://vitest.dev) — a blazing fast test runner designed for Vite — together with [@vue/test-utils](https://test-utils.vuejs.org/) to write unit tests that validate your component logic, props, events, and reactive behavior. - -This setup ensures your Vue.js components are tested in an environment that mirrors how users actually interact with your application. - ---- - -## Run tests during development - -`docker-vuejs-sample` application includes a sample test file at location: - -```console -$ src/components/__tests__/HelloWorld.spec.ts -``` - -This test uses Vitest and Vue Test Utils to verify the behavior of the HelloWorld component. - ---- - -### Step 1: Update compose.yaml - -Add a new service named `vuejs-test` to your `compose.yaml` file. This service allows you to run your test suite in an isolated containerized environment. - -```yaml {hl_lines="22-26",linenos=true} -services: - vuejs-prod: - build: - context: . - dockerfile: Dockerfile - image: docker-vuejs-sample - ports: - - "8080:8080" - - vuejs-dev: - build: - context: . - dockerfile: Dockerfile.dev - ports: - - "5173:5173" - develop: - watch: - - action: sync - path: . - target: /app - - vuejs-test: - build: - context: . - dockerfile: Dockerfile.dev - command: ["npm", "run", "test:unit"] -``` - -The vuejs-test service reuses the same `Dockerfile.dev` used for [development](develop.md) and overrides the default command to run tests with `npm run test`. This setup ensures a consistent test environment that matches your local development configuration. - - -After completing the previous steps, your project directory should contain the following files: - -```text -├── docker-vuejs-sample/ -│ ├── Dockerfile -│ ├── Dockerfile.dev -│ ├── .dockerignore -│ ├── compose.yaml -│ └── nginx.conf -``` - -### Step 2: Run the tests - -To execute your test suite inside the container, run the following command from your project root: - -```console -$ docker compose run --rm vuejs-test -``` - -This command will: -- Start the `vuejs-test` service defined in your `compose.yaml` file. -- Execute the `npm run test` script using the same environment as development. -- Automatically remove the container after the tests complete [`docker compose run --rm`](/reference/cli/docker/compose/run/) command. - -You should see output similar to the following: - -```shell -Test Files: 1 passed (1) -Tests: 1 passed (1) -Start at: 16:50:55 -Duration: 718ms -``` - -> [!NOTE] -> For more information about Compose commands, see the [Compose CLI -> reference](/reference/cli/docker/compose/). - ---- - -## Summary - -In this section, you learned how to run unit tests for your Vue.js application inside a Docker container using Vitest and Docker Compose. - -What you accomplished: -- Created a `vuejs-test` service in `compose.yaml` to isolate test execution. -- Reused the development `Dockerfile.dev` to ensure consistency between dev and test environments. -- Ran tests inside the container using `docker compose run --rm vuejs-test`. -- Ensured reliable, repeatable testing across environments without depending on your local machine setup. - ---- - -## Related resources - -Explore official references and best practices to sharpen your Docker testing workflow: - -- [Dockerfile reference](/reference/dockerfile/) – Understand all Dockerfile instructions and syntax. -- [Best practices for writing Dockerfiles](/develop/develop-images/dockerfile_best-practices/) – Write efficient, maintainable, and secure Dockerfiles. -- [Compose file reference](/compose/compose-file/) – Learn the full syntax and options available for configuring services in `compose.yaml`. -- [`docker compose run` CLI reference](/reference/cli/docker/compose/run/) – Run one-off commands in a service container. ---- - -## Next steps - -Next, you’ll learn how to set up a CI/CD pipeline using GitHub Actions to automatically build and test your Vue.js application in a containerized environment. This ensures your code is validated on every push or pull request, maintaining consistency and reliability across your development workflow. diff --git a/content/guides/wiremock.md b/content/guides/wiremock.md index 0913e9c08464..d6cc63eea425 100644 --- a/content/guides/wiremock.md +++ b/content/guides/wiremock.md @@ -4,9 +4,8 @@ description: &desc Mocking API services in development and testing with WireMock keywords: WireMock, container-supported development linktitle: Mocking API services with WireMock summary: *desc -tags: [app-dev, distributed-systems] -languages: [js] params: + tags: [testing] time: 20 minutes --- diff --git a/content/guides/zscaler/index.md b/content/guides/zscaler.md similarity index 98% rename from content/guides/zscaler/index.md rename to content/guides/zscaler.md index 8fbd51f6a1d5..c705e34e526a 100644 --- a/content/guides/zscaler/index.md +++ b/content/guides/zscaler.md @@ -1,12 +1,12 @@ --- title: Using Docker with Zscaler -tags: [networking, admin] summary: | - This guide explains how to embed Zscaler’s root certificate into Docker + This guide explains how to embed Zscaler's root certificate into Docker images, allowing containers to operate securely with Zscaler proxies and avoid SSL errors. keywords: zscaler, proxy, ssl certificates, corporate network, https, root certificate params: + tags: [admin] time: 10 minutes --- diff --git a/data/languages.yaml b/data/languages.yaml deleted file mode 100644 index c3cae399d134..000000000000 --- a/data/languages.yaml +++ /dev/null @@ -1,30 +0,0 @@ -c-sharp: - title: "C#" - icon: https://cdn.jsdelivr.net/gh/devicons/devicon@latest/icons/csharp/csharp-original.svg -cpp: - title: C++ - icon: https://cdn.jsdelivr.net/gh/devicons/devicon@latest/icons/cplusplus/cplusplus-original.svg -go: - title: Go - icon: https://cdn.jsdelivr.net/gh/devicons/devicon@latest/icons/go/go-original.svg -java: - title: Java - icon: https://cdn.jsdelivr.net/gh/devicons/devicon@latest/icons/java/java-original.svg -js: - title: JavaScript - icon: https://cdn.jsdelivr.net/gh/devicons/devicon@latest/icons/javascript/javascript-original.svg -php: - title: PHP - icon: https://cdn.jsdelivr.net/gh/devicons/devicon@latest/icons/php/php-original.svg -python: - title: Python - icon: https://cdn.jsdelivr.net/gh/devicons/devicon@latest/icons/python/python-original.svg -r: - title: R - icon: https://cdn.jsdelivr.net/gh/devicons/devicon@latest/icons/r/r-original.svg -ruby: - title: Ruby - icon: https://cdn.jsdelivr.net/gh/devicons/devicon@latest/icons/ruby/ruby-original.svg -rust: - title: Rust - icon: https://cdn.jsdelivr.net/gh/devicons/devicon@latest/icons/rust/rust-original.svg diff --git a/data/tags.yaml b/data/tags.yaml index cb5bccf7d9d6..71b447ea653a 100644 --- a/data/tags.yaml +++ b/data/tags.yaml @@ -1,42 +1,14 @@ admin: - title: Administration + title: Admin ai: title: AI -app-dev: - title: App development -best-practices: - title: Best practices -cloud-services: - title: Cloud services -data-science: - title: Data science +cicd: + title: CI/CD databases: title: Databases -deploy: +deployment: title: Deployment -devops: - title: DevOps -dhi: - title: Docker Hardened Images -distributed-systems: - title: Distributed systems -faq: - title: FAQ -frameworks: - title: Frameworks -labs: - title: Labs -networking: - title: Networking -observability: - title: Observability -product-demo: - title: Product demo -release-notes: - title: Release notes -secrets: - title: Secrets -testing-with-docker: - title: Testing with Docker -troubleshooting: - title: Troubleshooting +security: + title: Security +testing: + title: Testing From 522b727afc4201f1e15f4ae439325de3f7051445 Mon Sep 17 00:00:00 2001 From: David Karlsson <35727626+dvdksn@users.noreply.github.com> Date: Thu, 11 Jun 2026 17:38:14 +0200 Subject: [PATCH 2/8] =?UTF-8?q?guides:=20Phase=202=20=E2=80=94=20Vercel=20?= =?UTF-8?q?KB=20landing=20page=20+=20clean=20content=20layout?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Landing page (layouts/guides/landing.html): - Remove sidebar filter (checkboxes, language buttons) - Hero section: guide count, large "Guides." heading, description - Sticky left "Jump to" nav with numbered categories and counts - Sections per tag with 2-column card grid (title | summary) - Scrollspy via IntersectionObserver highlights active section - No Alpine.js filtering — browse by scrolling/jumping Content pages (layouts/guides/list.html, new): - Clean left sidebar: just mainnav, no guide-specific metadata - Slightly wider article (max-w-5xl) - TOC stays in right column via baseof.html default Other: - data/tags.yaml: add description to each tag (shown under section heading) - layouts/_partials/guide-languages.html: deleted (languages removed in Phase 1) - layouts/_partials/sidebar/guides.html: simplify (remove stepper, resource_links, languages — now just title, summary, tags, time, back link) Co-Authored-By: Claude Sonnet 4.6 --- content/guides/_index.md | 10 +- data/tags.yaml | 7 + layouts/_partials/guide-languages.html | 15 -- layouts/_partials/sidebar/guides.html | 34 +-- layouts/guides/landing.html | 340 +++++++------------------ layouts/guides/list.html | 9 + 6 files changed, 112 insertions(+), 303 deletions(-) delete mode 100644 layouts/_partials/guide-languages.html create mode 100644 layouts/guides/list.html diff --git a/content/guides/_index.md b/content/guides/_index.md index 100bdaf9b24a..9c4a04caaf0d 100644 --- a/content/guides/_index.md +++ b/content/guides/_index.md @@ -1,7 +1,7 @@ --- title: Docker guides linkTitle: Guides -description: Explore the Docker guides +description: Step-by-step tutorials, organized by what you're trying to do. keywords: docker, guides, tutorials, learning paths, getting started params: icon: book-open @@ -18,10 +18,8 @@ aliases: - /learning-paths/ --- -Explore our collection of guides to learn how Docker can optimize your -development workflows and how to use it with specific languages, frameworks, or -technologies. +Explore guides to learn how Docker can optimize your development workflows and +how to use it with specific languages, frameworks, or technologies. Can't find the guide you're looking for? Open an issue on the -[docker/docs](https://github.com/docker/docs/issues/new) repository to let us -know. +[docker/docs](https://github.com/docker/docs/issues/new) repository. diff --git a/data/tags.yaml b/data/tags.yaml index 71b447ea653a..89e8fd2d2eda 100644 --- a/data/tags.yaml +++ b/data/tags.yaml @@ -1,14 +1,21 @@ admin: title: Admin + description: Manage Docker organizations, users, and access controls. ai: title: AI + description: Build AI-powered apps, agents, and integrations with Docker. cicd: title: CI/CD + description: Automate builds, tests, and deployments in your pipelines. databases: title: Databases + description: Run and connect to databases in containers. deployment: title: Deployment + description: Deploy containerized apps to Kubernetes and other platforms. security: title: Security + description: Harden images, scan vulnerabilities, and secure your supply chain. testing: title: Testing + description: Integration testing with real databases and services. diff --git a/layouts/_partials/guide-languages.html b/layouts/_partials/guide-languages.html deleted file mode 100644 index 681f6d1ebfa1..000000000000 --- a/layouts/_partials/guide-languages.html +++ /dev/null @@ -1,15 +0,0 @@ -{{- range . -}} - {{- $lang := index hugo.Data.languages . -}} - {{- with $lang }} - - - {{ .title }} - - {{- end -}} -{{- end -}} diff --git a/layouts/_partials/sidebar/guides.html b/layouts/_partials/sidebar/guides.html index 584d9530f10d..b24a45b10ef6 100644 --- a/layouts/_partials/sidebar/guides.html +++ b/layouts/_partials/sidebar/guides.html @@ -3,37 +3,17 @@
    {{ $guide.Title }}
    -
    {{ $guide.Summary }}
    -
    -
    - {{- with $guide.Params.languages }} - {{ partial "guide-languages.html" . }} - {{- end }} - {{- with $guide.Params.tags }} - {{ partial "guide-tags.html" . }} - {{- end }} -
    +
    {{ $guide.Summary }}
    +
    + {{- with $guide.Params.tags }} + {{ partial "guide-tags.html" . }} + {{- end }} {{- with $guide.Params.time }} -
    - {{ partialCached "icon" "clock" "clock" }} +
    + {{ partialCached "icon" "clock" "clock" }} {{ . }}
    {{- end -}}
    - {{- if gt (len $guide.Pages) 0 }} - {{- partial "guides-stepper.html" . }} - {{- end }} - {{- with $guide.Params.resource_links }} -
    -

    Resources:

    - -
    - {{- end }} « Back to all guides
    diff --git a/layouts/guides/landing.html b/layouts/guides/landing.html index 18c93bf4eda1..0cef854cc970 100644 --- a/layouts/guides/landing.html +++ b/layouts/guides/landing.html @@ -1,273 +1,103 @@ {{ define "left" }} {{- partial "sidebar/mainnav.html" . }} -
    -

    Filter guides by tag or programming language.

    -

    -
    -
    Tags
    -
    - {{- range $name, $data := hugo.Data.tags }} - {{- if where $.Pages "Params.tags" "intersect" (slice $name) }} - {{- $id := anchorize (fmt.Printf "tag-%s" $name) }} -
    - - -
    - {{- end }} - {{ end }} -
    -
    Languages
    -
    - {{- range $name, $data := hugo.Data.languages }} - {{- if where $.Pages "Params.languages" "intersect" (slice $name) }} - {{- $id := anchorize (fmt.Printf "lang-%s" $name) }} - - {{- end }} - {{ end }} -
    -
    -
    {{ end }} {{ define "main" }} -
    -
    - {{- partial "breadcrumbs.html" . }} -

    {{ .Title }}

    - {{ .Content }} -
    + {{- partial "breadcrumbs.html" . }} - // Loop over the filters object - for (const [key, value] of Object.entries(this.filters)) { - if (!value || value.length === 0) continue; - - for (const g of guides) { - // Get the dataset for the current guide (e.g., languages or tags) - const terms = JSON.parse(g.dataset[key]); - - // Check if any of the filter values exist in the terms - const containsSome = terms.some(term => this.filters[key].includes(term)); - - // If none of the terms match the filter, mark the guide as hidden - if (!containsSome) { - hiddenSet.add(g.id) - } - } - } - - this.hidden = Array.from(hiddenSet) - }, +
    +

    + {{- len .Pages }} guides +

    +

    + Guides. +

    + {{- with .Description }} +

    {{ . }}

    + {{- end }} +
    +
    -
    -

    Featured guides

    -
    - {{- $featured := where .Pages "Params.featured" true }} - {{- with $featured }} - {{- range . }} -
    - - {{- $img := resources.Get (.Params.image | default "/images/thumbnail.webp") }} - -

    {{ .Title }}

    -     -

    {{ .Summary }}

    -
    - {{ template "guide-metadata" . }} -
    -
    - {{- end }} - {{- end }} -
    -
    -
    -

    - All guides -

    -
    -
    - {{ partialCached "icon" "funnel" "funnel" }} + +
    -
    - {{- range $name, $data := hugo.Data.tags }} -
    - {{ template "termchip" $data.title }} -
    - {{- end }} - {{- range $name, $data := hugo.Data.languages }} -
    - - {{ $data.title }} -
    - {{- end }} -
    -
    -
    - {{- range .Pages }} -
    + + {{- printf "%02d" (add $i 1) -}} + + {{ $tagData.title }} + + {{ len $pages }} + + {{- end }} + {{- end }} + + + +
    + {{- range $i, $tag := $tagOrder }} + {{- $tagData := index hugo.Data.tags $tag }} + {{- $pages := where $.Pages "Params.tags" "intersect" (slice $tag) }} + {{- if $pages }} +
    -
    - {{ .Title }} - {{ template "guide-metadata" . }} +
    +

    + {{- printf "%02d" (add $i 1) -}} +

    +

    {{ $tagData.title }}

    + {{- with $tagData.description }} +

    {{ . }}

    + {{- end }}
    -
    +
    + {{- range $pages }} +
    + {{ .Title }} +

    {{ .Summary }}

    +
    + {{- end }} +
    +
    {{- end }} -
    + {{- end }}
    -
    -
    -{{ end }} - -{{- define "guide-metadata" }} -
    -
    - {{- with .Params.languages }} - {{ partial "guide-languages.html" . }} - {{- end }} - {{- with .Params.tags }} - {{ partial "guide-tags.html" . }} - {{- end }}
    - {{- with .Params.time }} -
    - {{ partialCached "icon" "clock" "clock" }} - {{ . }} -
    - {{- end }}
    -{{- end }} - -{{- define "termchip" }} - - - {{ partialCached "icon" "tag" "tag" }} - - {{ . }} - -{{- end }} +{{ end }} diff --git a/layouts/guides/list.html b/layouts/guides/list.html new file mode 100644 index 000000000000..ddae5ce15443 --- /dev/null +++ b/layouts/guides/list.html @@ -0,0 +1,9 @@ +{{ define "left" }} + {{- partial "sidebar/mainnav.html" . }} +{{ end }} + +{{ define "article" }} +
    + {{ partial "content-default.html" . }} +
    +{{ end }} From 1a5c768dfc0f9b9673a90d90cb8afab508128689 Mon Sep 17 00:00:00 2001 From: David Karlsson <35727626+dvdksn@users.noreply.github.com> Date: Fri, 12 Jun 2026 10:11:42 +0200 Subject: [PATCH 3/8] guides: split cicd into languages + cicd, add labs group MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Tag taxonomy expanded from 7 to 9 tags: - languages (new): 17 language/framework guides pulled from cicd (angular, bun, cpp, deno, dotnet, golang, java, laravel, nextjs, nodejs, php, python, r, reactjs, ruby, rust, vuejs) - cicd: now pipeline/build tools only (gha, bake, azure-pipelines, docker-build-cloud, docker-compose, opentelemetry, etc.) - labs (new): 12 interactive lab guides previously scattered across ai, security, and cicd; now grouped together at the end Landing page tag order: languages, ai, testing, cicd, security, databases, deployment, admin, labs Also fixes carried over from build testing: - Broken front matter delimiters in container-supported-development.md and dhi-openshift.md (resource_links[]--- → newline before ---) - Orphaned url: lines in 16 guide files (leftover resource_links children) - 96 broken relative links in 18 merged guide _index.md files (containerize.md, configure-github-actions.md etc. → ./ or anchor) - /guides/docker-scout/sbom.md and s3c.md refs in sbom.md → /guides/docker-scout/ Co-Authored-By: Claude Sonnet 4.6 --- content/guides/admin-set-up/_index.md | 1 - content/guides/angular/_index.md | 14 +++++++------- content/guides/bun/_index.md | 12 ++++++------ .../guides/container-supported-development.md | 3 ++- content/guides/cpp/_index.md | 12 ++++++------ content/guides/deno/_index.md | 12 ++++++------ content/guides/dhi-openshift.md | 3 ++- content/guides/dotnet/_index.md | 18 +++++++++--------- content/guides/frameworks/laravel/_index.md | 3 +-- content/guides/genai-pdf-bot/_index.md | 2 +- .../guides/github-sonarqube-sandbox/_index.md | 1 - content/guides/golang/_index.md | 18 +++++++++--------- content/guides/java/_index.md | 14 +++++++------- content/guides/lab-agentic-apps.md | 3 +-- content/guides/lab-ai-fundamentals.md | 3 +-- content/guides/lab-attestation-basics.md | 3 +-- content/guides/lab-building-images.md | 3 +-- content/guides/lab-compose-quickstart.md | 3 +-- .../guides/lab-container-getting-started.md | 3 +-- .../lab-container-supported-development.md | 3 +-- content/guides/lab-containerized-sdlc.md | 3 +-- .../guides/lab-creating-ai-product-reviewer.md | 3 +-- content/guides/lab-dhi-node.md | 3 +-- content/guides/lab-docker-agent.md | 3 +-- content/guides/lab-mcp-gateway.md | 3 +-- content/guides/nextjs/_index.md | 14 +++++++------- content/guides/php/_index.md | 14 +++++++------- content/guides/python/_index.md | 18 +++++++++--------- content/guides/r/_index.md | 12 ++++++------ content/guides/rag-ollama/_index.md | 2 +- content/guides/reactjs/_index.md | 14 +++++++------- content/guides/ruby/_index.md | 12 ++++++------ content/guides/rust/_index.md | 12 ++++++------ content/guides/testcontainers-cloud/_index.md | 1 - content/guides/vuejs/_index.md | 14 +++++++------- .../build/metadata/attestations/sbom.md | 2 +- data/tags.yaml | 6 ++++++ layouts/guides/landing.html | 4 ++-- 38 files changed, 133 insertions(+), 141 deletions(-) diff --git a/content/guides/admin-set-up/_index.md b/content/guides/admin-set-up/_index.md index e77cb481df49..76663071c972 100644 --- a/content/guides/admin-set-up/_index.md +++ b/content/guides/admin-set-up/_index.md @@ -13,7 +13,6 @@ params: tags: [admin] time: 20 minutes image: - url: "https://www.docker.com/pricing?ref=Docs&refAction=DocsGuidesAdminSetup" --- diff --git a/content/guides/angular/_index.md b/content/guides/angular/_index.md index bd6c2fd1e59f..046e4d0c841d 100644 --- a/content/guides/angular/_index.md +++ b/content/guides/angular/_index.md @@ -12,7 +12,7 @@ aliases: - /guides/angular/develop/ - /guides/angular/run-tests/ params: - tags: [cicd] + tags: [languages] time: 20 minutes --- @@ -579,7 +579,7 @@ In the next section, you'll learn how to develop your application using Docker c ### Prerequisites -Complete [Containerize Angular application](containerize.md). +Complete [Containerize Angular application](./). --- @@ -751,7 +751,7 @@ In the next section, you'll learn how to run unit tests for your Angular applica ### Prerequisites -Complete all the previous sections of this guide, starting with [Containerize Angular application](containerize.md). +Complete all the previous sections of this guide, starting with [Containerize Angular application](./). ### Overview @@ -883,7 +883,7 @@ Next, you’ll learn how to set up a CI/CD pipeline using GitHub Actions to auto ### Prerequisites Before you begin, make sure you’ve completed the following: -- Complete all the previous sections of this guide, starting with [Containerize Angular application](containerize.md). +- Complete all the previous sections of this guide, starting with [Containerize Angular application](./). - [Enable Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. > **New to Kubernetes?** @@ -905,7 +905,7 @@ Follow these steps to define your deployment configuration: 2. Open the file in your IDE or preferred text editor. -3. Add the following configuration, and be sure to replace `{DOCKER_USERNAME}` and `{DOCKERHUB_PROJECT_NAME}` with your actual Docker Hub username and repository name from the previous [Automate your builds with GitHub Actions](configure-github-actions.md). +3. Add the following configuration, and be sure to replace `{DOCKER_USERNAME}` and `{DOCKERHUB_PROJECT_NAME}` with your actual Docker Hub username and repository name from the previous [Automate your builds with GitHub Actions](./). ```yaml @@ -957,7 +957,7 @@ This manifest defines two key Kubernetes resources, separated by `---`: - Deployment Deploys a single replica of your Angular application inside a pod. The pod uses the Docker image built and pushed by your GitHub Actions CI/CD workflow - (refer to [Automate your builds with GitHub Actions](configure-github-actions.md)). + (refer to [Automate your builds with GitHub Actions](./)). The container listens on port `8080`, which is typically used by [Nginx](https://nginx.org/en/docs/) to serve your production Angular app. - Service (NodePort) @@ -1077,7 +1077,7 @@ Explore official references and best practices to sharpen your Kubernetes deploy ### Prerequisites -Complete all the previous sections of this guide, starting with [Containerize an Angular application](containerize.md). +Complete all the previous sections of this guide, starting with [Containerize an Angular application](./). You must also have: - A [GitHub](https://github.com/signup) account. diff --git a/content/guides/bun/_index.md b/content/guides/bun/_index.md index 537c91740a62..3889eedb8188 100644 --- a/content/guides/bun/_index.md +++ b/content/guides/bun/_index.md @@ -11,7 +11,7 @@ aliases: - /guides/bun/deploy/ - /guides/bun/develop/ params: - tags: [cicd] + tags: [languages] time: 10 minutes --- @@ -210,7 +210,7 @@ containers. ### Prerequisites -Complete [Containerize a Bun application](containerize.md). +Complete [Containerize a Bun application](./). ### Overview @@ -278,7 +278,7 @@ In the next section, you'll take a look at how to set up a CI/CD pipeline using ### Prerequisites -Complete all the previous sections of this guide, starting with [Containerize a Bun application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. +Complete all the previous sections of this guide, starting with [Containerize a Bun application](./). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. ### Overview @@ -403,7 +403,7 @@ Next, learn how you can locally test and debug your workloads on Kubernetes befo ### Prerequisites -- Complete all the previous sections of this guide, starting with [Containerize a Bun application](containerize.md). +- Complete all the previous sections of this guide, starting with [Containerize a Bun application](./). - [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. ### Overview @@ -416,7 +416,7 @@ In your `bun-docker` directory, create a file named `docker-kubernetes.yml`. Open the file in an IDE or text editor and add the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker username and the name of the repository that you created in [Configure CI/CD for -your Bun application](configure-ci-cd.md). +your Bun application](./). ```yaml apiVersion: apps/v1 @@ -460,7 +460,7 @@ In this Kubernetes YAML file, there are two objects, separated by the `---`: you'll get just one replica, or copy of your pod. That pod, which is described under `template`, has just one container in it. The container is created from the image built by GitHub Actions in [Configure CI/CD for - your Bun application](configure-ci-cd.md). + your Bun application](./). - A NodePort service, which will route traffic from port 30001 on your host to port 3000 inside the pods it routes to, allowing you to reach your app from the network. diff --git a/content/guides/container-supported-development.md b/content/guides/container-supported-development.md index 80dc6dc36e80..a5239165a053 100644 --- a/content/guides/container-supported-development.md +++ b/content/guides/container-supported-development.md @@ -10,7 +10,8 @@ params: tags: [cicd] image: images/learning-paths/container-supported-development.png time: 20 minutes - resource_links: []--- + resource_links: [] +--- Containers offer a consistent way to build, share, and run applications across different environments. While containers are typically used to containerize your application, they also make it incredibly easy to run essential services needed for development. Instead of installing or connecting to a remote database, you can easily launch your own database. But the possibilities don't stop there. diff --git a/content/guides/cpp/_index.md b/content/guides/cpp/_index.md index 16bf60e9101e..8cb478ea36f0 100644 --- a/content/guides/cpp/_index.md +++ b/content/guides/cpp/_index.md @@ -19,7 +19,7 @@ aliases: - /guides/cpp/multistage/ - /guides/cpp/security/ params: - tags: [cicd] + tags: [languages] time: 20 minutes --- @@ -234,7 +234,7 @@ containers. ### Prerequisites -Complete [Containerize a C++ application](containerize.md). +Complete [Containerize a C++ application](./). ### Overview @@ -303,7 +303,7 @@ In the next section, you'll take a look at how to set up a CI/CD pipeline using ### Prerequisites -Complete all the previous sections of this guide, starting with [Containerize a C++ application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. +Complete all the previous sections of this guide, starting with [Containerize a C++ application](./). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. ### Overview @@ -428,7 +428,7 @@ Next, learn how you can locally test and debug your workloads on Kubernetes befo ### Prerequisites -- Complete all the previous sections of this guide, starting with [Containerize a C++ application](containerize.md). +- Complete all the previous sections of this guide, starting with [Containerize a C++ application](./). - [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. ### Overview @@ -441,7 +441,7 @@ In your `c-plus-plus-docker` directory, create a file named `docker-kubernetes.yml`. Open the file in an IDE or text editor and add the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker username and the name of the repository that you created in [Configure CI/CD for -your C++ application](configure-ci-cd.md). +your C++ application](./). ```yaml apiVersion: apps/v1 @@ -485,7 +485,7 @@ In this Kubernetes YAML file, there are two objects, separated by the `---`: you'll get just one replica, or copy of your pod. That pod, which is described under `template`, has just one container in it. The container is created from the image built by GitHub Actions in [Configure CI/CD for - your C++ application](configure-ci-cd.md). + your C++ application](./). - A NodePort service, which will route traffic from port 30001 on your host to port 8080 inside the pods it routes to, allowing you to reach your app from the network. diff --git a/content/guides/deno/_index.md b/content/guides/deno/_index.md index 79686d78ed68..38d9d7eba7bb 100644 --- a/content/guides/deno/_index.md +++ b/content/guides/deno/_index.md @@ -11,7 +11,7 @@ aliases: - /guides/deno/deploy/ - /guides/deno/develop/ params: - tags: [cicd] + tags: [languages] time: 10 minutes --- @@ -233,7 +233,7 @@ containers. ### Prerequisites -Complete [Containerize a Deno application](containerize.md). +Complete [Containerize a Deno application](./). ### Overview @@ -301,7 +301,7 @@ In the next section, you'll take a look at how to set up a CI/CD pipeline using ### Prerequisites -Complete all the previous sections of this guide, starting with [Containerize a Deno application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. +Complete all the previous sections of this guide, starting with [Containerize a Deno application](./). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. ### Overview @@ -425,7 +425,7 @@ Next, learn how you can locally test and debug your workloads on Kubernetes befo ### Prerequisites -- Complete all the previous sections of this guide, starting with [Containerize a Deno application](containerize.md). +- Complete all the previous sections of this guide, starting with [Containerize a Deno application](./). - [Turn on Kubernetes](/manuals//desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. ### Overview @@ -438,7 +438,7 @@ In your `deno-docker` directory, create a file named `docker-kubernetes.yml`. Open the file in an IDE or text editor and add the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker username and the name of the repository that you created in [Configure CI/CD for -your Deno application](configure-ci-cd.md). +your Deno application](./). ```yaml apiVersion: apps/v1 @@ -482,7 +482,7 @@ In this Kubernetes YAML file, there are two objects, separated by the `---`: you'll get just one replica, or copy of your pod. That pod, which is described under `template`, has just one container in it. The container is created from the image built by GitHub Actions in [Configure CI/CD for - your Deno application](configure-ci-cd.md). + your Deno application](./). - A NodePort service, which will route traffic from port 30001 on your host to port 8000 inside the pods it routes to, allowing you to reach your app from the network. diff --git a/content/guides/dhi-openshift.md b/content/guides/dhi-openshift.md index 57d9295cb017..2885d0985ab0 100644 --- a/content/guides/dhi-openshift.md +++ b/content/guides/dhi-openshift.md @@ -11,7 +11,8 @@ params: - An OpenShift cluster (version 4.11 or later recommended) - The oc CLI authenticated to your cluster - A Docker Hub account with access to Docker Hardened Images - - Familiarity with OpenShift Security Context Constraints (SCCs)--- + - Familiarity with OpenShift Security Context Constraints (SCCs) +--- Docker Hardened Images (DHI) can be deployed on Red Hat OpenShift Container Platform, but OpenShift’s security model differs from standard Kubernetes in diff --git a/content/guides/dotnet/_index.md b/content/guides/dotnet/_index.md index ea2e2e3bf0a2..9eec457ccfcc 100644 --- a/content/guides/dotnet/_index.md +++ b/content/guides/dotnet/_index.md @@ -18,7 +18,7 @@ aliases: - /guides/dotnet/develop/ - /guides/dotnet/run-tests/ params: - tags: [cicd] + tags: [languages] time: 20 minutes toc_min: 1 toc_max: 2 @@ -395,7 +395,7 @@ Docker containers. ### Prerequisites -Complete [Containerize a .NET application](containerize.md). +Complete [Containerize a .NET application](./). ### Overview @@ -410,9 +410,9 @@ In this section, you'll learn how to set up a development environment for your c This section uses a different branch of the `docker-dotnet-sample` repository that contains an updated .NET application. The updated application is on the `add-db` branch of the repository you cloned in [Containerize a .NET -application](containerize.md). +application](./). -To get the updated code, you need to checkout the `add-db` branch. For the changes you made in [Containerize a .NET application](containerize.md), for this section, you can stash them. In a terminal, run the following commands in the `docker-dotnet-sample` directory. +To get the updated code, you need to checkout the `add-db` branch. For the changes you made in [Containerize a .NET application](./), for this section, you can stash them. In a terminal, run the following commands in the `docker-dotnet-sample` directory. 1. Stash any previous changes. @@ -812,7 +812,7 @@ In the next section, you'll learn how to run unit tests using Docker. ### Prerequisites -Complete all the previous sections of this guide, starting with [Containerize a .NET application](containerize.md). +Complete all the previous sections of this guide, starting with [Containerize a .NET application](./). ### Overview @@ -953,7 +953,7 @@ Next, you’ll learn how to set up a CI/CD pipeline using GitHub Actions. ### Prerequisites -Complete all the previous sections of this guide, starting with [Containerize a .NET application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. +Complete all the previous sections of this guide, starting with [Containerize a .NET application](./). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. ### Overview @@ -1094,7 +1094,7 @@ Next, learn how you can locally test and debug your workloads on Kubernetes befo ### Prerequisites - Complete all the previous sections of this guide, starting with [Containerize - a .NET application](containerize.md). + a .NET application](./). - [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. @@ -1111,7 +1111,7 @@ In your `docker-dotnet-sample` directory, create a file named `docker-dotnet-kubernetes.yaml`. Open the file in an IDE or text editor and add the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker username and the name of the repository that you created in [Configure CI/CD for -your .NET application](configure-ci-cd.md). +your .NET application](./). ```yaml apiVersion: apps/v1 @@ -1230,7 +1230,7 @@ In this Kubernetes YAML file, there are four objects, separated by the `---`. In you'll get just one replica, or copy of your pod. That pod, which is described under `template`, has just one container in it. The container is created from the image built by GitHub Actions in [Configure CI/CD for your - .NET application](configure-ci-cd.md). + .NET application](./). - A NodePort service, which will route traffic from port 30001 on your host to port 8080 inside the pods it routes to, allowing you to reach your app from the network. diff --git a/content/guides/frameworks/laravel/_index.md b/content/guides/frameworks/laravel/_index.md index aaa7846c9524..d12886cc783f 100644 --- a/content/guides/frameworks/laravel/_index.md +++ b/content/guides/frameworks/laravel/_index.md @@ -11,9 +11,8 @@ aliases: - /guides/frameworks/laravel/prerequisites/ - /guides/frameworks/laravel/production-setup/ params: - tags: [cicd] + tags: [languages] time: 30 minutes - url: https://github.com/dockersamples/laravel-docker-examples --- diff --git a/content/guides/genai-pdf-bot/_index.md b/content/guides/genai-pdf-bot/_index.md index 469055455934..f844f616510b 100644 --- a/content/guides/genai-pdf-bot/_index.md +++ b/content/guides/genai-pdf-bot/_index.md @@ -283,7 +283,7 @@ In the next section, you'll learn how you can run your application, database, an ### Prerequisites -Complete [Containerize a generative AI application](containerize.md). +Complete [Containerize a generative AI application](./). ### Overview diff --git a/content/guides/github-sonarqube-sandbox/_index.md b/content/guides/github-sonarqube-sandbox/_index.md index f2dbb69bac43..8756972aafe1 100644 --- a/content/guides/github-sonarqube-sandbox/_index.md +++ b/content/guides/github-sonarqube-sandbox/_index.md @@ -12,7 +12,6 @@ params: tags: [cicd] time: 40 minutes image: - url: https://docs.docker.com/ai/mcp-catalog-and-toolkit/sandboxes/ --- diff --git a/content/guides/golang/_index.md b/content/guides/golang/_index.md index cc3ea119d17d..0705b5caedf7 100644 --- a/content/guides/golang/_index.md +++ b/content/guides/golang/_index.md @@ -25,7 +25,7 @@ aliases: - /guides/golang/run-containers/ - /guides/golang/run-tests/ params: - tags: [cicd] + tags: [languages] time: 30 minutes --- @@ -552,7 +552,7 @@ In the next module, you’ll take a look at how to run your image as a container ### Prerequisites -Work through the steps to containerize a Go application in [Build your Go image](build-images.md). +Work through the steps to containerize a Go application in [Build your Go image](./). ### Overview @@ -753,7 +753,7 @@ In this module, you learned how to run containers and publish ports. You also le ### Prerequisites -Work through the steps of the [run your image as a container](run-containers.md) module to learn how to manage the lifecycle of your containers. +Work through the steps of the [run your image as a container](./) module to learn how to manage the lifecycle of your containers. ### Introduction @@ -1477,7 +1477,7 @@ In the next module, you'll take a look at one possible approach to running funct ### Prerequisites -Complete the [Build your Go image](build-images.md) section of this guide. +Complete the [Build your Go image](./) section of this guide. ### Overview @@ -1487,7 +1487,7 @@ tests and end-to-end testing. In this guide you take a look at running your unit tests in Docker when building. For this section, use the `docker-gs-ping` project that you cloned in [Build -your Go image](build-images.md). +your Go image](./). ### Run tests when building @@ -1564,7 +1564,7 @@ you’ll learn how to set up a CI/CD pipeline using GitHub Actions. ### Prerequisites -Complete the previous sections of this guide, starting with [Build your Go image](build-images.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. +Complete the previous sections of this guide, starting with [Build your Go image](./). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. ### Overview @@ -1690,7 +1690,7 @@ Next, learn how you can locally test and debug your workloads on Kubernetes befo ### Prerequisites - Complete all the previous sections of this guide, starting with [Build - your Go image](build-images.md). + your Go image](./). - [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. @@ -1707,7 +1707,7 @@ In your project directory, create a file named `docker-go-kubernetes.yaml`. Open the file in an IDE or text editor and add the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker username and the name of the repository that you created in [Configure CI/CD for -your Go application](configure-ci-cd.md). +your Go application](./). ```yaml apiVersion: apps/v1 @@ -1839,7 +1839,7 @@ In this Kubernetes YAML file, there are four objects, separated by the `---`. In you'll get just one replica, or copy of your pod. That pod, which is described under `template`, has just one container in it. The container is created from the image built by GitHub Actions in [Configure CI/CD for your - Go application](configure-ci-cd.md). + Go application](./). - A NodePort service, which will route traffic from port 30001 on your host to port 8080 inside the pods it routes to, allowing you to reach your app from the network. diff --git a/content/guides/java/_index.md b/content/guides/java/_index.md index 96d1da579d21..9c8c50be5565 100644 --- a/content/guides/java/_index.md +++ b/content/guides/java/_index.md @@ -21,7 +21,7 @@ aliases: - /guides/java/develop/ - /guides/java/run-tests/ params: - tags: [cicd] + tags: [languages] time: 20 minutes --- @@ -320,7 +320,7 @@ Docker containers. ### Prerequisites -Work through the steps to containerize your application in [Containerize your app](containerize.md). +Work through the steps to containerize your application in [Containerize your app](./). ### Overview @@ -715,7 +715,7 @@ In the next section, you’ll take a look at how to run unit tests in Docker. ### Prerequisites -Complete all the previous sections of this guide, starting with [Containerize a Java application](containerize.md). +Complete all the previous sections of this guide, starting with [Containerize a Java application](./). ### Overview @@ -832,7 +832,7 @@ GitHub Actions. ### Prerequisites -Complete the previous sections of this guide, starting with [Containerize your app](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. +Complete the previous sections of this guide, starting with [Containerize your app](./). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. ### Overview @@ -967,7 +967,7 @@ Next, learn how you can locally test and debug your workloads on Kubernetes befo ### Prerequisites -- Complete all the previous sections of this guide, starting with [Containerize your app](containerize.md). +- Complete all the previous sections of this guide, starting with [Containerize your app](./). - [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. ### Overview @@ -983,7 +983,7 @@ In your `spring-petclinic` directory, create a file named `docker-java-kubernetes.yaml`. Open the file in an IDE or text editor and add the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker username and the name of the repository that you created in [Configure CI/CD for -your Java application](configure-ci-cd.md). +your Java application](./). ```yaml apiVersion: apps/v1 @@ -1027,7 +1027,7 @@ In this Kubernetes YAML file, there are two objects, separated by the `---`: you'll get just one replica, or copy of your pod. That pod, which is described under `template`, has just one container in it. The container is created from the image built by GitHub Actions in [Configure CI/CD for - your Java application](configure-ci-cd.md). + your Java application](./). - A NodePort service, which will route traffic from port 30001 on your host to port 8080 inside the pods it routes to, allowing you to reach your app from the network. diff --git a/content/guides/lab-agentic-apps.md b/content/guides/lab-agentic-apps.md index 5839ecd44bcc..19bc988a8c51 100644 --- a/content/guides/lab-agentic-apps.md +++ b/content/guides/lab-agentic-apps.md @@ -11,9 +11,8 @@ keywords: AI, Docker, Model Runner, MCP Gateway, agentic apps, lab, labspace aliases: - /labs/docker-for-ai/agentic-apps/ params: - tags: [ai] + tags: [labs] time: 20 minutes - url: https://github.com/dockersamples/labspace-agentic-apps-with-docker --- Get up and running with building agentic applications using Compose, Docker diff --git a/content/guides/lab-ai-fundamentals.md b/content/guides/lab-ai-fundamentals.md index 01163b62c1e6..5ee8718201ad 100644 --- a/content/guides/lab-ai-fundamentals.md +++ b/content/guides/lab-ai-fundamentals.md @@ -12,9 +12,8 @@ keywords: AI, Docker, Model Runner, prompt engineering, RAG, tool calling, lab, aliases: - /labs/docker-for-ai/ai-fundamentals/ params: - tags: [ai] + tags: [labs] time: 45 minutes - url: https://github.com/dockersamples/labspace-ai-fundamentals --- Get hands-on with the four core pillars of AI application development: models, diff --git a/content/guides/lab-attestation-basics.md b/content/guides/lab-attestation-basics.md index 3560d7a53fe4..4153c114ac08 100644 --- a/content/guides/lab-attestation-basics.md +++ b/content/guides/lab-attestation-basics.md @@ -10,9 +10,8 @@ summary: | attach OpenVEX statements to declare vulnerability exploitability status. keywords: Docker, supply chain, SBOM, provenance, SLSA, Cosign, VEX, attestations, security, lab, labspace params: - tags: [security] + tags: [labs] time: 45 minutes - url: https://github.com/dockersamples/labspace-attestation-basics --- Prove where your container images came from and that they haven't been diff --git a/content/guides/lab-building-images.md b/content/guides/lab-building-images.md index 6eeef74b76c7..424d5df7a274 100644 --- a/content/guides/lab-building-images.md +++ b/content/guides/lab-building-images.md @@ -11,9 +11,8 @@ summary: | base image selection, and build secrets. keywords: Docker, Dockerfile, images, multi-stage builds, layer caching, build secrets, lab, labspace params: - tags: [cicd] + tags: [labs] time: 45 minutes - url: https://github.com/dockersamples/labspace-building-images --- Take a working but naïve Dockerfile and progressively improve it into a diff --git a/content/guides/lab-compose-quickstart.md b/content/guides/lab-compose-quickstart.md index 67f4a43b582e..97d821d91a0f 100644 --- a/content/guides/lab-compose-quickstart.md +++ b/content/guides/lab-compose-quickstart.md @@ -11,9 +11,8 @@ summary: | with watch mode, data persistence, and modular Compose file composition. keywords: Docker, Compose, multi-container, Flask, Redis, watch mode, volumes, lab, labspace params: - tags: [cicd] + tags: [labs] time: 45 minutes - url: https://github.com/dockersamples/labspace-compose-quickstart --- Build a Python Flask and Redis hit-counter app using Docker Compose, starting diff --git a/content/guides/lab-container-getting-started.md b/content/guides/lab-container-getting-started.md index fb40a4884a03..3e8ad4ad83b9 100644 --- a/content/guides/lab-container-getting-started.md +++ b/content/guides/lab-container-getting-started.md @@ -9,9 +9,8 @@ summary: | image from a Node.js app, and optionally push it to Docker Hub. keywords: Docker, containers, Dockerfile, images, getting started, lab, labspace params: - tags: [cicd] + tags: [labs] time: 30 minutes - url: https://github.com/dockersamples/labspace-container-getting-started --- Start from zero and learn Docker's core building blocks. You'll run pre-built diff --git a/content/guides/lab-container-supported-development.md b/content/guides/lab-container-supported-development.md index 1ee08231940b..1915b40a888e 100644 --- a/content/guides/lab-container-supported-development.md +++ b/content/guides/lab-container-supported-development.md @@ -11,9 +11,8 @@ summary: | visualizer — all without installing anything on the host. keywords: Docker, Compose, local development, PostgreSQL, pgAdmin, containers, lab, labspace params: - tags: [cicd] + tags: [labs] time: 30 minutes - url: https://github.com/dockersamples/labspace-container-supported-development --- Use containers to run the services your app depends on — databases, caches, diff --git a/content/guides/lab-containerized-sdlc.md b/content/guides/lab-containerized-sdlc.md index b4f68a1ecc4e..4acd08027520 100644 --- a/content/guides/lab-containerized-sdlc.md +++ b/content/guides/lab-containerized-sdlc.md @@ -11,9 +11,8 @@ summary: | with containers at every stage of the SDLC. keywords: Docker, Compose, Testcontainers, Kubernetes, CI/CD, SDLC, lab, labspace params: - tags: [cicd] + tags: [labs] time: 60 minutes - url: https://github.com/dockersamples/labspace-containerized-sdlc --- Build a real Node.js API, then apply containers at every stage of the software diff --git a/content/guides/lab-creating-ai-product-reviewer.md b/content/guides/lab-creating-ai-product-reviewer.md index c1476d1f1e15..bb295c1119d0 100644 --- a/content/guides/lab-creating-ai-product-reviewer.md +++ b/content/guides/lab-creating-ai-product-reviewer.md @@ -13,9 +13,8 @@ keywords: AI, Docker, Model Runner, sentiment analysis, embeddings, RAG, lab, la aliases: - /labs/docker-for-ai/creating-ai-product-reviewer/ params: - tags: [ai] + tags: [labs] time: 60 minutes - url: https://github.com/dockersamples/labspace-creating-ai-product-reviewer --- Build a complete feedback analysis pipeline for a fictional AI product called diff --git a/content/guides/lab-dhi-node.md b/content/guides/lab-dhi-node.md index 6906195c00ef..9d1a1f51c274 100644 --- a/content/guides/lab-dhi-node.md +++ b/content/guides/lab-dhi-node.md @@ -11,9 +11,8 @@ summary: | builds with DHI, and explore SBOMs, VEX, and compliance attestations. keywords: Docker, Hardened Images, DHI, Node.js, Docker Scout, CVE, security, SBOM, lab, labspace params: - tags: [security] + tags: [labs] time: 30 minutes - url: https://github.com/dockersamples/labspace-dhi-node --- Migrate a Node.js application from a standard `node:24-trixie-slim` base image diff --git a/content/guides/lab-docker-agent.md b/content/guides/lab-docker-agent.md index 4fce0f713515..6e21c6111d66 100644 --- a/content/guides/lab-docker-agent.md +++ b/content/guides/lab-docker-agent.md @@ -12,9 +12,8 @@ aliases: - /labs/docker-for-ai/cagent/ - /guides/lab-cagent/ params: - tags: [ai] + tags: [labs] time: 20 minutes - url: https://github.com/ajeetraina/labspace-cagent --- This lab walks you through building intelligent agents with Docker Agent. You'll learn beginner diff --git a/content/guides/lab-mcp-gateway.md b/content/guides/lab-mcp-gateway.md index 28b6cc81ef5a..4d2b97720702 100644 --- a/content/guides/lab-mcp-gateway.md +++ b/content/guides/lab-mcp-gateway.md @@ -11,9 +11,8 @@ keywords: AI, Docker, MCP, MCP Gateway, MCP servers, lab, labspace aliases: - /labs/docker-for-ai/mcp-gateway/ params: - tags: [ai] + tags: [labs] time: 20 minutes - url: https://github.com/dockersamples/labspace-mcp-gateway --- This lab provides a comprehensive, hands-on overview of the Docker MCP Gateway, diff --git a/content/guides/nextjs/_index.md b/content/guides/nextjs/_index.md index 736dbc397230..def9d81ffd33 100644 --- a/content/guides/nextjs/_index.md +++ b/content/guides/nextjs/_index.md @@ -14,7 +14,7 @@ aliases: - /guides/nextjs/develop/ - /guides/nextjs/run-tests/ params: - tags: [cicd] + tags: [languages] time: 20 minutes --- @@ -1016,7 +1016,7 @@ In the next section, you'll learn how to develop your application using Docker c ### Prerequisites -Complete [Containerize Next.js application](containerize.md). +Complete [Containerize Next.js application](./). --- @@ -1219,7 +1219,7 @@ In the next section, you'll learn how to run unit tests for your Next.js applica ### Prerequisites -Complete all the previous sections of this guide, starting with [Containerize Next.js application](containerize.md). +Complete all the previous sections of this guide, starting with [Containerize Next.js application](./). ### Overview @@ -1480,7 +1480,7 @@ Next, you'll learn how to set up a CI/CD pipeline using GitHub Actions to automa ### Prerequisites -Complete all the previous sections of this guide, starting with [Containerize Next.js application](containerize.md). +Complete all the previous sections of this guide, starting with [Containerize Next.js application](./). You must also have: - A [GitHub](https://github.com/signup) account. @@ -1807,7 +1807,7 @@ Next, learn how you can locally test and debug your Next.js workloads on Kuberne ### Prerequisites Before you begin, make sure you've completed the following: -- Complete all the previous sections of this guide, starting with [Containerize Next.js application](containerize.md). +- Complete all the previous sections of this guide, starting with [Containerize Next.js application](./). - [Enable Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. > **New to Kubernetes?** @@ -1829,7 +1829,7 @@ Follow these steps to define your deployment configuration: 2. Open the file in your IDE or preferred text editor. -3. Add the following configuration, and be sure to replace `{DOCKER_USERNAME}` and `{DOCKERHUB_PROJECT_NAME}` with your actual Docker Hub username and repository name from the previous [Automate your builds with GitHub Actions](configure-github-actions.md). +3. Add the following configuration, and be sure to replace `{DOCKER_USERNAME}` and `{DOCKERHUB_PROJECT_NAME}` with your actual Docker Hub username and repository name from the previous [Automate your builds with GitHub Actions](./). ```yaml @@ -1879,7 +1879,7 @@ This manifest defines two key Kubernetes resources, separated by `---`: - Deployment Deploys a single replica of your Next.js application inside a pod. The pod uses the Docker image built and pushed by your GitHub Actions CI/CD workflow - (refer to [Automate your builds with GitHub Actions](configure-github-actions.md)). + (refer to [Automate your builds with GitHub Actions](./)). The container listens on port `3000`, which is the default port for Next.js applications. - Service (NodePort) diff --git a/content/guides/php/_index.md b/content/guides/php/_index.md index 394197735138..3fb1085d5704 100644 --- a/content/guides/php/_index.md +++ b/content/guides/php/_index.md @@ -19,7 +19,7 @@ aliases: - /guides/php/develop/ - /guides/php/run-tests/ params: - tags: [cicd] + tags: [languages] time: 20 minutes --- @@ -315,7 +315,7 @@ Docker containers. ### Prerequisites -Complete [Containerize a PHP application](containerize.md). +Complete [Containerize a PHP application](./). ### Overview @@ -765,7 +765,7 @@ In the next section, you'll learn how to run unit tests using Docker. ### Prerequisites -Complete all the previous sections of this guide, starting with [Containerize a PHP application](containerize.md). +Complete all the previous sections of this guide, starting with [Containerize a PHP application](./). ### Overview @@ -875,7 +875,7 @@ Next, you’ll learn how to set up a CI/CD pipeline using GitHub Actions. ### Prerequisites -Complete all the previous sections of this guide, starting with [Containerize a PHP application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. +Complete all the previous sections of this guide, starting with [Containerize a PHP application](./). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. ### Overview @@ -1016,7 +1016,7 @@ Next, learn how you can locally test and debug your workloads on Kubernetes befo ### Prerequisites - Complete all the previous sections of this guide, starting with [Containerize - a PHP application](containerize.md). + a PHP application](./). - [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. @@ -1033,7 +1033,7 @@ In your `docker-php-sample` directory, create a file named `docker-php-kubernetes.yaml`. Open the file in an IDE or text editor and add the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker username and the name of the repository that you created in [Configure CI/CD for -your PHP application](configure-ci-cd.md). +your PHP application](./). ```yaml apiVersion: apps/v1 @@ -1077,7 +1077,7 @@ In this Kubernetes YAML file, there are two objects, separated by the `---`: you'll get just one replica, or copy of your pod. That pod, which is described under `template`, has just one container in it. The container is created from the image built by GitHub Actions in [Configure CI/CD for your - PHP application](configure-ci-cd.md). + PHP application](./). - A NodePort service, which will route traffic from port 30001 on your host to port 80 inside the pods it routes to, allowing you to reach your app from the network. diff --git a/content/guides/python/_index.md b/content/guides/python/_index.md index b2f5e0133f17..d0f8417c3c4c 100644 --- a/content/guides/python/_index.md +++ b/content/guides/python/_index.md @@ -22,7 +22,7 @@ aliases: - /guides/python/lint-format-typing/ - /guides/python/secure-supply-chain/ params: - tags: [cicd] + tags: [languages] time: 20 minutes --- @@ -476,7 +476,7 @@ In the next section, you'll take a look at how to set up a local development env ### Prerequisites -Complete [Containerize a Python application](containerize.md). +Complete [Containerize a Python application](./). ### Overview @@ -2031,7 +2031,7 @@ In the next section, you'll learn how you can set up linting, formatting, and ty Complete [Develop your app](develop.md). This topic requires a local Python installation because the tools and Git hooks introduced here run on your host. If you don't want to install Python locally, skip this topic. The same -checks run in CI in the [next topic](configure-github-actions.md). +checks run in CI in the [next topic](./). ### Overview @@ -2209,7 +2209,7 @@ Related information: ### Next steps -- [Configure GitHub Actions](configure-github-actions.md) to run these checks automatically +- [Configure GitHub Actions](./) to run these checks automatically - Customize linting rules to match your team's style preferences - Explore advanced type checking features @@ -2217,7 +2217,7 @@ Related information: ### Prerequisites -Complete all the previous sections of this guide, starting with [Containerize a Python application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. +Complete all the previous sections of this guide, starting with [Containerize a Python application](./). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. If you didn't create a [GitHub repository](https://github.com/new) for your project yet, it is time to do it. After creating the repository, don't forget to [add a remote](https://docs.github.com/en/get-started/getting-started-with-git/managing-remote-repositories) and ensure you can commit and [push your code](https://docs.github.com/en/get-started/using-git/pushing-commits-to-a-remote-repository#about-git-push) to GitHub. @@ -2362,13 +2362,13 @@ Related information: ### Next steps In the next section, you'll learn how to inspect and generate supply chain -attestations for your image. See [Secure your supply chain](secure-supply-chain.md). +attestations for your image. See [Secure your supply chain](./). ## Secure your Python image supply chain ### Prerequisites -Complete [Configure CI/CD for your Python application](configure-github-actions.md). +Complete [Configure CI/CD for your Python application](./). ### Overview @@ -2542,7 +2542,7 @@ Create the following two Kubernetes manifest files in your `python-docker-example` directory. Before applying `docker-python-kubernetes.yaml`, replace `DOCKER_USERNAME/REPO_NAME` with your Docker username and the repository name that you created in [Configure CI/CD for -your Python application](./configure-github-actions.md). +your Python application](./). {{< files name="python-docker-example" >}} @@ -2700,7 +2700,7 @@ In these Kubernetes YAML files, there are various objects, separated by the `--- you'll get just one replica, or copy of your pod. That pod, which is described under `template`, has just one container in it. The container is created from the image built by GitHub Actions in [Configure CI/CD for - your Python application](configure-github-actions.md). + your Python application](./). - A Service, which will define how the ports are mapped in the containers. - A PersistentVolumeClaim, to define a storage that will be persistent through restarts for the database. - A Secret, which stores the database password as a Kubernetes Secret resource. diff --git a/content/guides/r/_index.md b/content/guides/r/_index.md index 3c4e6c494750..8c3b92639474 100644 --- a/content/guides/r/_index.md +++ b/content/guides/r/_index.md @@ -19,7 +19,7 @@ aliases: - /guides/r/deploy/ - /guides/r/develop/ params: - tags: [cicd] + tags: [languages] time: 10 minutes --- @@ -127,7 +127,7 @@ containers. ### Prerequisites -Complete [Containerize a R application](containerize.md). +Complete [Containerize a R application](./). ### Overview @@ -340,7 +340,7 @@ In the next section, you'll take a look at how to set up a CI/CD pipeline using ### Prerequisites -Complete all the previous sections of this guide, starting with [Containerize a R application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. +Complete all the previous sections of this guide, starting with [Containerize a R application](./). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. ### Overview @@ -465,7 +465,7 @@ Next, learn how you can locally test and debug your workloads on Kubernetes befo ### Prerequisites -- Complete all the previous sections of this guide, starting with [Containerize a R application](containerize.md). +- Complete all the previous sections of this guide, starting with [Containerize a R application](./). - [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. ### Overview @@ -478,7 +478,7 @@ In your `r-docker-dev` directory, create a file named `docker-r-kubernetes.yaml`. Open the file in an IDE or text editor and add the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker username and the name of the repository that you created in [Configure CI/CD for -your R application](configure-ci-cd.md). +your R application](./). ```yaml apiVersion: apps/v1 @@ -525,7 +525,7 @@ In this Kubernetes YAML file, there are two objects, separated by the `---`: you'll get just one replica, or copy of your pod. That pod, which is described under `template`, has just one container in it. The container is created from the image built by GitHub Actions in [Configure CI/CD for - your R application](configure-ci-cd.md). + your R application](./). - A NodePort service, which will route traffic from port 30001 on your host to port 3838 inside the pods it routes to, allowing you to reach your app from the network. diff --git a/content/guides/rag-ollama/_index.md b/content/guides/rag-ollama/_index.md index 2bd3e2e325bf..e8f68ca10b9f 100644 --- a/content/guides/rag-ollama/_index.md +++ b/content/guides/rag-ollama/_index.md @@ -127,7 +127,7 @@ In the next section, you'll learn how to properly configure the application with ### Prerequisites -Complete [Containerize a RAG application](containerize.md). +Complete [Containerize a RAG application](./). ### Overview diff --git a/content/guides/reactjs/_index.md b/content/guides/reactjs/_index.md index 18feceef58a1..59c6b24f5261 100644 --- a/content/guides/reactjs/_index.md +++ b/content/guides/reactjs/_index.md @@ -12,7 +12,7 @@ aliases: - /guides/reactjs/develop/ - /guides/reactjs/run-tests/ params: - tags: [cicd] + tags: [languages] time: 20 minutes --- @@ -558,7 +558,7 @@ In the next section, you'll learn how to develop your application using Docker c ### Prerequisites -Complete [Containerize React.js application](containerize.md). +Complete [Containerize React.js application](./). --- @@ -757,7 +757,7 @@ In the next section, you'll learn how to run unit tests for your React.js applic ### Prerequisites -Complete all the previous sections of this guide, starting with [Containerize React.js application](containerize.md). +Complete all the previous sections of this guide, starting with [Containerize React.js application](./). ### Overview @@ -931,7 +931,7 @@ Next, you’ll learn how to set up a CI/CD pipeline using GitHub Actions to auto ### Prerequisites Before you begin, make sure you’ve completed the following: -- Complete all the previous sections of this guide, starting with [Containerize React.js application](containerize.md). +- Complete all the previous sections of this guide, starting with [Containerize React.js application](./). - [Enable Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. > **New to Kubernetes?** @@ -953,7 +953,7 @@ Follow these steps to define your deployment configuration: 2. Open the file in your IDE or preferred text editor. -3. Add the following configuration, and be sure to replace `{DOCKER_USERNAME}` and `{DOCKERHUB_PROJECT_NAME}` with your actual Docker Hub username and repository name from the previous [Automate your builds with GitHub Actions](configure-github-actions.md). +3. Add the following configuration, and be sure to replace `{DOCKER_USERNAME}` and `{DOCKERHUB_PROJECT_NAME}` with your actual Docker Hub username and repository name from the previous [Automate your builds with GitHub Actions](./). ```yaml @@ -998,7 +998,7 @@ This manifest defines two key Kubernetes resources, separated by `---`: - Deployment Deploys a single replica of your React.js application inside a pod. The pod uses the Docker image built and pushed by your GitHub Actions CI/CD workflow - (refer to [Automate your builds with GitHub Actions](configure-github-actions.md)). + (refer to [Automate your builds with GitHub Actions](./)). The container listens on port `8080`, which is typically used by [Nginx](https://nginx.org/en/docs/) to serve your production React app. - Service (NodePort) @@ -1118,7 +1118,7 @@ Explore official references and best practices to sharpen your Kubernetes deploy ### Prerequisites -Complete all the previous sections of this guide, starting with [Containerize React.js application](containerize.md). +Complete all the previous sections of this guide, starting with [Containerize React.js application](./). You must also have: - A [GitHub](https://github.com/signup) account. diff --git a/content/guides/ruby/_index.md b/content/guides/ruby/_index.md index 58fad9d67b5d..7c2934711cca 100644 --- a/content/guides/ruby/_index.md +++ b/content/guides/ruby/_index.md @@ -21,7 +21,7 @@ aliases: - /guides/ruby/deploy/ - /guides/ruby/develop/ params: - tags: [cicd] + tags: [languages] time: 20 minutes --- @@ -420,7 +420,7 @@ In the next section, you'll take a look at how to set up a CI/CD pipeline using ### Prerequisites -Complete all the previous sections of this guide, starting with [Containerize a Ruby on Rails application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. +Complete all the previous sections of this guide, starting with [Containerize a Ruby on Rails application](./). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. If you didn't create a [GitHub repository](https://github.com/new) for your project yet, it is time to do it. After creating the repository, don't forget to [add a remote](https://docs.github.com/en/get-started/getting-started-with-git/managing-remote-repositories) and ensure you can commit and [push your code](https://docs.github.com/en/get-started/using-git/pushing-commits-to-a-remote-repository#about-git-push) to GitHub. @@ -521,7 +521,7 @@ In the next section, you'll learn how you can develop your application using con ### Prerequisites -Complete [Containerize a Ruby on Rails application](containerize.md). +Complete [Containerize a Ruby on Rails application](./). ### Overview @@ -716,7 +716,7 @@ In the next section, you'll learn how you can locally test and debug your worklo ### Prerequisites -- Complete all the previous sections of this guide, starting with [Containerize a Ruby on Rails application](containerize.md). +- Complete all the previous sections of this guide, starting with [Containerize a Ruby on Rails application](./). - [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. ### Overview @@ -729,7 +729,7 @@ In your `docker-ruby-on-rails` directory, create a file named `docker-ruby-on-rails-kubernetes.yaml`. Open the file in an IDE or text editor and add the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker username and the name of the repository that you created in [Configure CI/CD for -your Ruby on Rails application](configure-github-actions.md). +your Ruby on Rails application](./). ```yaml apiVersion: apps/v1 @@ -773,7 +773,7 @@ In this Kubernetes YAML file, there are two objects, separated by the `---`: you'll get just one replica, or copy of your pod. That pod, which is described under `template`, has just one container in it. The container is created from the image built by GitHub Actions in [Configure CI/CD for - your Ruby on Rails application](configure-github-actions.md). + your Ruby on Rails application](./). - A NodePort service, which will route traffic from port 30001 on your host to port 8001 inside the pods it routes to, allowing you to reach your app from the network. diff --git a/content/guides/rust/_index.md b/content/guides/rust/_index.md index 62291a1ed9ae..0093138def41 100644 --- a/content/guides/rust/_index.md +++ b/content/guides/rust/_index.md @@ -19,7 +19,7 @@ aliases: - /guides/rust/develop/ - /guides/rust/run-containers/ params: - tags: [cicd] + tags: [languages] time: 20 minutes --- @@ -344,7 +344,7 @@ In the next section learn how to run your image as a container. ### Prerequisite -You have completed [Build your Rust image](build-images.md) and you have built an image. +You have completed [Build your Rust image](./) and you have built an image. ### Overview @@ -354,7 +354,7 @@ To run an image inside of a container, you use the `docker run` command. The `do ### Run an image -Use `docker run` to run the image you built in [Build your Rust image](build-images.md). +Use `docker run` to run the image you built in [Build your Rust image](./). ```console $ docker run docker-rust-image-dhi @@ -790,7 +790,7 @@ In the cloned repository's directory, create a new directory named `db` and insi mysecretpassword ``` -If you have any other containers running from the previous sections, [stop](./run-containers.md#stop-start-and-name-containers) them now. +If you have any other containers running from the previous sections, [stop](#stop-start-and-name-containers) them now. Now, run the following `docker compose up` command to start your application. @@ -966,7 +966,7 @@ In your `docker-rust-postgres` directory, create a file named `docker-rust-kubernetes.yaml`. Open the file in an IDE or text editor and add the following contents. Replace `DOCKER_USERNAME/REPO_NAME` with your Docker username and the name of the repository that you created in [Configure CI/CD for -your Rust application](configure-ci-cd.md). +your Rust application](./). ```yaml apiVersion: apps/v1 @@ -1100,7 +1100,7 @@ In this Kubernetes YAML file, there are four objects, separated by the `---`. In you'll get just one replica, or copy of your pod. That pod, which is described under `template`, has just one container in it. The container is created from the image built by GitHub Actions in [Configure CI/CD for your - Rust application](configure-ci-cd.md). + Rust application](./). - A NodePort service, which will route traffic from port 30001 on your host to port 5000 inside the pods it routes to, allowing you to reach your app from the network. diff --git a/content/guides/testcontainers-cloud/_index.md b/content/guides/testcontainers-cloud/_index.md index c47b205f4eaa..409a9fc5c734 100644 --- a/content/guides/testcontainers-cloud/_index.md +++ b/content/guides/testcontainers-cloud/_index.md @@ -16,7 +16,6 @@ params: tags: [testing] image: images/learning-paths/testcontainers-cloud-learning-path.png time: 12 minutes - url: https://www.docker.com/search/?_sf_s=testcontainers%20cloud --- diff --git a/content/guides/vuejs/_index.md b/content/guides/vuejs/_index.md index 3db4bc43e261..c38d5a23defe 100644 --- a/content/guides/vuejs/_index.md +++ b/content/guides/vuejs/_index.md @@ -13,7 +13,7 @@ aliases: - /guides/vuejs/develop/ - /guides/vuejs/run-tests/ params: - tags: [cicd] + tags: [languages] time: 20 minutes --- @@ -579,7 +579,7 @@ In the next section, you'll learn how to develop your application using Docker c ### Prerequisites -Complete [Containerize Vue.js application](containerize.md). +Complete [Containerize Vue.js application](./). --- @@ -762,7 +762,7 @@ In the next section, you'll learn how to run unit tests for your Vue.js applicat ### Prerequisites -Complete all the previous sections of this guide, starting with [Containerize Vue.js application](containerize.md). +Complete all the previous sections of this guide, starting with [Containerize Vue.js application](./). ### Overview @@ -895,7 +895,7 @@ Next, you’ll learn how to set up a CI/CD pipeline using GitHub Actions to auto ### Prerequisites Before you begin, make sure you’ve completed the following: -- Complete all the previous sections of this guide, starting with [Containerize Vue.js application](containerize.md). +- Complete all the previous sections of this guide, starting with [Containerize Vue.js application](./). - [Enable Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. > **New to Kubernetes?** @@ -917,7 +917,7 @@ Follow these steps to define your deployment configuration: 2. Open the file in your IDE or preferred text editor. -3. Add the following configuration, and be sure to replace `{DOCKER_USERNAME}` and `{DOCKERHUB_PROJECT_NAME}` with your actual Docker Hub username and repository name from the previous [Automate your builds with GitHub Actions](configure-github-actions.md). +3. Add the following configuration, and be sure to replace `{DOCKER_USERNAME}` and `{DOCKERHUB_PROJECT_NAME}` with your actual Docker Hub username and repository name from the previous [Automate your builds with GitHub Actions](./). ```yaml @@ -969,7 +969,7 @@ This manifest defines two key Kubernetes resources, separated by `---`: - Deployment Deploys a single replica of your Vue.js application inside a pod. The pod uses the Docker image built and pushed by your GitHub Actions CI/CD workflow - (refer to [Automate your builds with GitHub Actions](configure-github-actions.md)). + (refer to [Automate your builds with GitHub Actions](./)). The container listens on port `8080`, which is typically used by [Nginx](https://nginx.org/en/docs/) to serve your production Vue.js app. - Service (NodePort) @@ -1089,7 +1089,7 @@ Explore official references and best practices to sharpen your Kubernetes deploy ### Prerequisites -Complete all the previous sections of this guide, starting with [Containerize an Vue.js application](containerize.md). +Complete all the previous sections of this guide, starting with [Containerize an Vue.js application](./). You must also have: - A [GitHub](https://github.com/signup) account. diff --git a/content/manuals/build/metadata/attestations/sbom.md b/content/manuals/build/metadata/attestations/sbom.md index ee34dc895665..8a9a99d62a17 100644 --- a/content/manuals/build/metadata/attestations/sbom.md +++ b/content/manuals/build/metadata/attestations/sbom.md @@ -7,7 +7,7 @@ aliases: - /build/attestations/sbom/ --- -SBOM attestations help ensure [software supply chain transparency](/guides/docker-scout/s3c.md) by verifying the software artifacts an image contains and the artifacts used to create the image. Metadata included in an [SBOM](/guides/docker-scout/sbom.md) for describing software artifacts may include: +SBOM attestations help ensure [software supply chain transparency](/guides/docker-scout/) by verifying the software artifacts an image contains and the artifacts used to create the image. Metadata included in an [SBOM](/guides/docker-scout/) for describing software artifacts may include: - Name of the artifact - Version diff --git a/data/tags.yaml b/data/tags.yaml index 89e8fd2d2eda..ab067f824068 100644 --- a/data/tags.yaml +++ b/data/tags.yaml @@ -13,6 +13,12 @@ databases: deployment: title: Deployment description: Deploy containerized apps to Kubernetes and other platforms. +labs: + title: Hands-on labs + description: Interactive, guided labs you can run directly in your browser. +languages: + title: Languages & frameworks + description: Containerize and develop apps in your language of choice. security: title: Security description: Harden images, scan vulnerabilities, and secure your supply chain. diff --git a/layouts/guides/landing.html b/layouts/guides/landing.html index 0cef854cc970..62ec9c900199 100644 --- a/layouts/guides/landing.html +++ b/layouts/guides/landing.html @@ -3,7 +3,7 @@ {{ end }} {{ define "main" }} - {{- $tagOrder := slice "ai" "testing" "cicd" "security" "databases" "deployment" "admin" }} + {{- $tagOrder := slice "languages" "ai" "testing" "cicd" "security" "databases" "deployment" "admin" "labs" }}
    {{- partial "breadcrumbs.html" . }} @@ -26,7 +26,7 @@
    {{ partial "github-links.html" . }}
    diff --git a/layouts/_partials/content-default.html b/layouts/_partials/content-default.html index 48b333564e25..9d7c32067066 100644 --- a/layouts/_partials/content-default.html +++ b/layouts/_partials/content-default.html @@ -1,5 +1,10 @@ {{ partial "breadcrumbs.html" . }}

    {{ .Title | safeHTML }}

    +{{- if and (eq .Type "guides") .Params.summary }} +

    + {{ .Params.summary }} +

    +{{- end }} diff --git a/layouts/baseof.html b/layouts/baseof.html index 062ff2bec1b7..0aa658176e28 100644 --- a/layouts/baseof.html +++ b/layouts/baseof.html @@ -30,7 +30,7 @@ } } })" - class="bg-background-toc dark:bg-background-toc fixed top-0 z-40 hidden h-screen w-full flex-none overflow-x-hidden overflow-y-auto md:sticky md:top-16 md:z-auto md:block md:h-[calc(100vh-64px)] md:w-[320px]" + class="bg-background-toc dark:bg-background-toc fixed top-0 z-40 hidden h-screen w-full flex-none overflow-x-hidden overflow-y-auto{{ if ne .Type "guides" }} md:sticky md:top-16 md:z-auto md:block md:h-[calc(100vh-64px)] md:w-[320px]{{ end }}" :class="{ 'hidden': ! $store.showSidebar }" > diff --git a/layouts/guides/list.html b/layouts/guides/list.html index ddae5ce15443..5d7ab8ed1ff3 100644 --- a/layouts/guides/list.html +++ b/layouts/guides/list.html @@ -2,8 +2,13 @@ {{- partial "sidebar/mainnav.html" . }} {{ end }} -{{ define "article" }} -
    - {{ partial "content-default.html" . }} -
    +{{ define "main" }} +
    +
    + {{ partial "content-default.html" . }} +
    + +
    {{ end }} diff --git a/layouts/guides/single.html b/layouts/guides/single.html new file mode 100644 index 000000000000..5d7ab8ed1ff3 --- /dev/null +++ b/layouts/guides/single.html @@ -0,0 +1,14 @@ +{{ define "left" }} + {{- partial "sidebar/mainnav.html" . }} +{{ end }} + +{{ define "main" }} +
    +
    + {{ partial "content-default.html" . }} +
    + +
    +{{ end }} From 6a9905012e4eac471b03bb618bf0428ac3bea267 Mon Sep 17 00:00:00 2001 From: David Karlsson <35727626+dvdksn@users.noreply.github.com> Date: Fri, 12 Jun 2026 12:38:30 +0200 Subject: [PATCH 5/8] guides: add text filter to the landing page - Instant client-side filter (Alpine.js) matching title, summary, and tags across the already-rendered guide list; multi-word queries are AND-matched - Refined search input with inline clear button and Esc-to-clear - While filtering: hide the jump-to rail and go single-column full-width, flatten the grouped sections into one list, and show a "N of M guides" meta bar with a Clear action - Container fills the viewport (min-height) so a single result no longer collapses the page - Whole-row link cards with hover affordance (arrow), plus a polished empty-results state Co-Authored-By: Claude Opus 4.8 (1M context) --- layouts/guides/landing.html | 148 ++++++++++++++++++++++++++++-------- 1 file changed, 118 insertions(+), 30 deletions(-) diff --git a/layouts/guides/landing.html b/layouts/guides/landing.html index 62ec9c900199..8ca413f5964a 100644 --- a/layouts/guides/landing.html +++ b/layouts/guides/landing.html @@ -5,11 +5,30 @@ {{ define "main" }} {{- $tagOrder := slice "languages" "ai" "testing" "cicd" "security" "databases" "deployment" "admin" "labs" }} -
    +
    {{- partial "breadcrumbs.html" . }} -
    -

    +

    +

    {{- len .Pages }} guides

    -
    - -
    From fe8c2af3b9fc38520f6260ee15b2e87cf0f084c5 Mon Sep 17 00:00:00 2001 From: David Karlsson <35727626+dvdksn@users.noreply.github.com> Date: Fri, 12 Jun 2026 12:51:48 +0200 Subject: [PATCH 6/8] guides: make collapsed guides single pages, recollapse refreshed nodejs - Convert every directory-based guide from a branch bundle (_index.md = section) to a leaf bundle (index.md = single page). URLs and any co-located images are preserved; the guides are now kind=page, not sections. - Re-collapse the Node.js guide on top of main's refresh (#25319): merge the refreshed containerize/develop/run-tests/configure-github-actions/ secure-supply-chain/deploy topics into one page, drop the languages field, move tags under params ([languages]), and add aliases for the old /guides/nodejs// URLs. - Landing: enumerate guides with RegularPagesRecursive instead of .Pages so every guide is listed (now that they're regular pages), including the nested frameworks/laravel guide that the non-recursive .Pages missed. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../admin-set-up/{_index.md => index.md} | 0 .../{_index.md => index.md} | 0 .../guides/angular/{_index.md => index.md} | 0 content/guides/bun/{_index.md => index.md} | 0 content/guides/cpp/{_index.md => index.md} | 0 content/guides/deno/{_index.md => index.md} | 0 .../{_index.md => index.md} | 0 .../docker-compose/{_index.md => index.md} | 0 .../docker-scout/{_index.md => index.md} | 0 content/guides/dotnet/{_index.md => index.md} | 0 .../laravel/{_index.md => index.md} | 0 .../genai-pdf-bot/{_index.md => index.md} | 0 .../{_index.md => index.md} | 0 .../{_index.md => index.md} | 0 content/guides/golang/{_index.md => index.md} | 0 content/guides/java/{_index.md => index.md} | 0 content/guides/nextjs/{_index.md => index.md} | 0 content/guides/nodejs/_index.md | 44 - .../guides/nodejs/configure-github-actions.md | 135 -- content/guides/nodejs/containerize.md | 413 ---- content/guides/nodejs/deploy.md | 274 --- content/guides/nodejs/develop.md | 633 ------ content/guides/nodejs/index.md | 1868 +++++++++++++++++ content/guides/nodejs/run-tests.md | 264 --- content/guides/nodejs/secure-supply-chain.md | 141 -- content/guides/php/{_index.md => index.md} | 0 .../guides/postgresql/{_index.md => index.md} | 0 content/guides/python/{_index.md => index.md} | 0 content/guides/r/{_index.md => index.md} | 0 .../guides/rag-ollama/{_index.md => index.md} | 0 .../guides/reactjs/{_index.md => index.md} | 0 content/guides/ros2/{_index.md => index.md} | 0 content/guides/ruby/{_index.md => index.md} | 0 content/guides/rust/{_index.md => index.md} | 0 .../{_index.md => index.md} | 0 .../{_index.md => index.md} | 0 .../{_index.md => index.md} | 0 .../{_index.md => index.md} | 0 .../{_index.md => index.md} | 0 .../{_index.md => index.md} | 0 .../{_index.md => index.md} | 0 .../{_index.md => index.md} | 0 .../{_index.md => index.md} | 0 .../{_index.md => index.md} | 0 .../{_index.md => index.md} | 0 .../{_index.md => index.md} | 0 .../{_index.md => index.md} | 0 .../{_index.md => index.md} | 0 .../{_index.md => index.md} | 0 .../{_index.md => index.md} | 0 .../{_index.md => index.md} | 0 .../{_index.md => index.md} | 0 .../{_index.md => index.md} | 0 .../{_index.md => index.md} | 0 content/guides/vuejs/{_index.md => index.md} | 0 layouts/guides/landing.html | 8 +- 56 files changed, 1872 insertions(+), 1908 deletions(-) rename content/guides/admin-set-up/{_index.md => index.md} (100%) rename content/guides/admin-user-management/{_index.md => index.md} (100%) rename content/guides/angular/{_index.md => index.md} (100%) rename content/guides/bun/{_index.md => index.md} (100%) rename content/guides/cpp/{_index.md => index.md} (100%) rename content/guides/deno/{_index.md => index.md} (100%) rename content/guides/docker-build-cloud/{_index.md => index.md} (100%) rename content/guides/docker-compose/{_index.md => index.md} (100%) rename content/guides/docker-scout/{_index.md => index.md} (100%) rename content/guides/dotnet/{_index.md => index.md} (100%) rename content/guides/frameworks/laravel/{_index.md => index.md} (100%) rename content/guides/genai-pdf-bot/{_index.md => index.md} (100%) rename content/guides/github-sonarqube-sandbox/{_index.md => index.md} (100%) rename content/guides/go-prometheus-monitoring/{_index.md => index.md} (100%) rename content/guides/golang/{_index.md => index.md} (100%) rename content/guides/java/{_index.md => index.md} (100%) rename content/guides/nextjs/{_index.md => index.md} (100%) delete mode 100644 content/guides/nodejs/_index.md delete mode 100644 content/guides/nodejs/configure-github-actions.md delete mode 100644 content/guides/nodejs/containerize.md delete mode 100644 content/guides/nodejs/deploy.md delete mode 100644 content/guides/nodejs/develop.md create mode 100644 content/guides/nodejs/index.md delete mode 100644 content/guides/nodejs/run-tests.md delete mode 100644 content/guides/nodejs/secure-supply-chain.md rename content/guides/php/{_index.md => index.md} (100%) rename content/guides/postgresql/{_index.md => index.md} (100%) rename content/guides/python/{_index.md => index.md} (100%) rename content/guides/r/{_index.md => index.md} (100%) rename content/guides/rag-ollama/{_index.md => index.md} (100%) rename content/guides/reactjs/{_index.md => index.md} (100%) rename content/guides/ros2/{_index.md => index.md} (100%) rename content/guides/ruby/{_index.md => index.md} (100%) rename content/guides/rust/{_index.md => index.md} (100%) rename content/guides/testcontainers-cloud/{_index.md => index.md} (100%) rename content/guides/testcontainers-dotnet-aspnet-core/{_index.md => index.md} (100%) rename content/guides/testcontainers-dotnet-getting-started/{_index.md => index.md} (100%) rename content/guides/testcontainers-go-getting-started/{_index.md => index.md} (100%) rename content/guides/testcontainers-java-aws-localstack/{_index.md => index.md} (100%) rename content/guides/testcontainers-java-getting-started/{_index.md => index.md} (100%) rename content/guides/testcontainers-java-jooq-flyway/{_index.md => index.md} (100%) rename content/guides/testcontainers-java-keycloak-spring-boot/{_index.md => index.md} (100%) rename content/guides/testcontainers-java-lifecycle/{_index.md => index.md} (100%) rename content/guides/testcontainers-java-micronaut-kafka/{_index.md => index.md} (100%) rename content/guides/testcontainers-java-micronaut-wiremock/{_index.md => index.md} (100%) rename content/guides/testcontainers-java-mockserver/{_index.md => index.md} (100%) rename content/guides/testcontainers-java-quarkus/{_index.md => index.md} (100%) rename content/guides/testcontainers-java-replace-h2/{_index.md => index.md} (100%) rename content/guides/testcontainers-java-service-configuration/{_index.md => index.md} (100%) rename content/guides/testcontainers-java-spring-boot-kafka/{_index.md => index.md} (100%) rename content/guides/testcontainers-java-spring-boot-rest-api/{_index.md => index.md} (100%) rename content/guides/testcontainers-java-wiremock/{_index.md => index.md} (100%) rename content/guides/testcontainers-nodejs-getting-started/{_index.md => index.md} (100%) rename content/guides/testcontainers-python-getting-started/{_index.md => index.md} (100%) rename content/guides/vuejs/{_index.md => index.md} (100%) diff --git a/content/guides/admin-set-up/_index.md b/content/guides/admin-set-up/index.md similarity index 100% rename from content/guides/admin-set-up/_index.md rename to content/guides/admin-set-up/index.md diff --git a/content/guides/admin-user-management/_index.md b/content/guides/admin-user-management/index.md similarity index 100% rename from content/guides/admin-user-management/_index.md rename to content/guides/admin-user-management/index.md diff --git a/content/guides/angular/_index.md b/content/guides/angular/index.md similarity index 100% rename from content/guides/angular/_index.md rename to content/guides/angular/index.md diff --git a/content/guides/bun/_index.md b/content/guides/bun/index.md similarity index 100% rename from content/guides/bun/_index.md rename to content/guides/bun/index.md diff --git a/content/guides/cpp/_index.md b/content/guides/cpp/index.md similarity index 100% rename from content/guides/cpp/_index.md rename to content/guides/cpp/index.md diff --git a/content/guides/deno/_index.md b/content/guides/deno/index.md similarity index 100% rename from content/guides/deno/_index.md rename to content/guides/deno/index.md diff --git a/content/guides/docker-build-cloud/_index.md b/content/guides/docker-build-cloud/index.md similarity index 100% rename from content/guides/docker-build-cloud/_index.md rename to content/guides/docker-build-cloud/index.md diff --git a/content/guides/docker-compose/_index.md b/content/guides/docker-compose/index.md similarity index 100% rename from content/guides/docker-compose/_index.md rename to content/guides/docker-compose/index.md diff --git a/content/guides/docker-scout/_index.md b/content/guides/docker-scout/index.md similarity index 100% rename from content/guides/docker-scout/_index.md rename to content/guides/docker-scout/index.md diff --git a/content/guides/dotnet/_index.md b/content/guides/dotnet/index.md similarity index 100% rename from content/guides/dotnet/_index.md rename to content/guides/dotnet/index.md diff --git a/content/guides/frameworks/laravel/_index.md b/content/guides/frameworks/laravel/index.md similarity index 100% rename from content/guides/frameworks/laravel/_index.md rename to content/guides/frameworks/laravel/index.md diff --git a/content/guides/genai-pdf-bot/_index.md b/content/guides/genai-pdf-bot/index.md similarity index 100% rename from content/guides/genai-pdf-bot/_index.md rename to content/guides/genai-pdf-bot/index.md diff --git a/content/guides/github-sonarqube-sandbox/_index.md b/content/guides/github-sonarqube-sandbox/index.md similarity index 100% rename from content/guides/github-sonarqube-sandbox/_index.md rename to content/guides/github-sonarqube-sandbox/index.md diff --git a/content/guides/go-prometheus-monitoring/_index.md b/content/guides/go-prometheus-monitoring/index.md similarity index 100% rename from content/guides/go-prometheus-monitoring/_index.md rename to content/guides/go-prometheus-monitoring/index.md diff --git a/content/guides/golang/_index.md b/content/guides/golang/index.md similarity index 100% rename from content/guides/golang/_index.md rename to content/guides/golang/index.md diff --git a/content/guides/java/_index.md b/content/guides/java/index.md similarity index 100% rename from content/guides/java/_index.md rename to content/guides/java/index.md diff --git a/content/guides/nextjs/_index.md b/content/guides/nextjs/index.md similarity index 100% rename from content/guides/nextjs/_index.md rename to content/guides/nextjs/index.md diff --git a/content/guides/nodejs/_index.md b/content/guides/nodejs/_index.md deleted file mode 100644 index bf73b585988b..000000000000 --- a/content/guides/nodejs/_index.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -title: Node.js language-specific guide -linkTitle: Node.js -description: Containerize and develop Node.js applications using Docker -keywords: getting started, node, node.js -summary: | - This guide explains how to containerize Node.js applications using Docker. -toc_min: 1 -toc_max: 2 -aliases: - - /language/nodejs/ - - /guides/language/nodejs/ -languages: [js] -tags: [dhi] -params: - time: 20 minutes ---- - -[Node.js](https://nodejs.org/en) is a JavaScript runtime for building server-side applications. This guide shows you how to containerize a TypeScript Node.js application using Docker, starting from a simple Express API and progressively adding features like a database and CI/CD. - -This guide focuses on a backend Node.js API. If you're building a standalone frontend application, Docker has dedicated guides for [React.js](/guides/reactjs/), [Vue.js](/guides/vuejs/), [Angular](/guides/angular/), and [Next.js](/guides/nextjs/). - -> **Acknowledgment** -> -> Docker thanks [Kristiyan Velkov](https://www.linkedin.com/in/kristiyan-velkov-763130b3/) for his contribution to this guide. - -## What will you learn? - -In this guide, you'll learn how to: - -- Containerize and run a Node.js application using Docker. -- Set up a local development environment using containers. -- Run tests inside a Docker container. -- Configure GitHub Actions for CI/CD with Docker. -- Inspect and generate supply chain attestations for your image. -- Deploy your containerized Node.js application to Kubernetes. - -Start by containerizing a Node.js application. - -## Prerequisites - -- Basic understanding of [JavaScript](https://developer.mozilla.org/en-US/docs/Web/JavaScript) and [TypeScript](https://www.typescriptlang.org/). -- Basic knowledge of [Node.js](https://nodejs.org/en) and [npm](https://docs.npmjs.com/about-npm). -- Familiarity with Docker concepts such as images, containers, and Dockerfiles. If you're new to Docker, start with the [Docker basics](/get-started/docker-concepts/the-basics/what-is-a-container.md) guide. diff --git a/content/guides/nodejs/configure-github-actions.md b/content/guides/nodejs/configure-github-actions.md deleted file mode 100644 index cd448ec88234..000000000000 --- a/content/guides/nodejs/configure-github-actions.md +++ /dev/null @@ -1,135 +0,0 @@ ---- -title: Automate your builds with GitHub Actions -linkTitle: GitHub Actions CI -weight: 40 -keywords: CI/CD, GitHub Actions, Node.js, Docker -description: Learn how to configure CI/CD using GitHub Actions for your Node.js application. -aliases: - - /language/nodejs/configure-ci-cd/ - - /guides/language/nodejs/configure-ci-cd/ ---- - -## Prerequisites - -Complete all the previous sections of this guide, starting with [Containerize a Node.js application](containerize.md). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. - -If you haven't created a [GitHub repository](https://github.com/new) for your project yet, do that now. After creating the repository, [add a remote](https://docs.github.com/en/get-started/getting-started-with-git/managing-remote-repositories) and make sure you can commit and [push your code](https://docs.github.com/en/get-started/using-git/pushing-commits-to-a-remote-repository#about-git-push) to GitHub. - -1. In your project's GitHub repository, open **Settings**, and go to **Secrets and variables** > **Actions**. - -2. Under the **Variables** tab, create a new **Repository variable** named `DOCKER_USERNAME` with your Docker ID as the value. - -3. Create a new [Personal Access Token (PAT)](/manuals/security/access-tokens.md#create-an-access-token) for Docker Hub. You can name this token `docker-tutorial`. Make sure access permissions include Read and Write. - -4. Add the PAT as a **Repository secret** in your GitHub repository, with the name `DOCKERHUB_TOKEN`. - -## Overview - -GitHub Actions is a CI/CD automation tool built into GitHub. A workflow is a YAML file that tells GitHub which jobs to run when something happens in your repository, like a push to a branch or a pull request opening. Workflows live in the `.github/workflows/` directory of your repository. - -In this section, you'll add a workflow that runs your tests on every push to the main branch, then builds your Docker image and pushes it to Docker Hub. - -## Define the GitHub Actions workflow - -You can create a GitHub Actions workflow by creating a YAML file in the `.github/workflows/` directory of your repository. Use your favorite text editor or the GitHub web interface. - -If you prefer to use the GitHub web interface: - -1. Go to your repository on GitHub and select the **Actions** tab. - -2. Select **set up a workflow yourself**. - - This takes you to a page for creating a new GitHub Actions workflow file in your repository. By default, the file is created under `.github/workflows/main.yml`. Change the filename to `build.yml`. - -If you prefer to use your text editor, create a new file named `build.yml` in the `.github/workflows/` directory of your repository. - -Add the following content to the file: - -{{< files name="nodejs-docker-example" >}} - -{{< file path=".github/workflows/build.yml" status="new" >}} -```yaml -# GitHub Actions workflow that runs on every push to main. -# - test: runs Vitest unit tests inside a container. -# - build_and_push: signs in to Docker Hub and the DHI registry, then -# builds and pushes the image. -name: Build and push Docker image - -on: - push: - branches: - - main - -jobs: - test: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@{{% param "checkout_action_version" %}} - - - name: Run tests - run: docker build --target test -t nodejs-test . && docker run --rm nodejs-test - - build_and_push: - runs-on: ubuntu-latest - needs: test - steps: - - uses: actions/checkout@{{% param "checkout_action_version" %}} - - - name: Login to Docker Hub - uses: docker/login-action@{{% param "login_action_version" %}} - with: - username: ${{ vars.DOCKER_USERNAME }} - password: ${{ secrets.DOCKERHUB_TOKEN }} - - - name: Login to Docker Hardened Images - uses: docker/login-action@{{% param "login_action_version" %}} - with: - registry: dhi.io - username: ${{ vars.DOCKER_USERNAME }} - password: ${{ secrets.DOCKERHUB_TOKEN }} - - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} - - - name: Build and push - uses: docker/build-push-action@{{% param "build_push_action_version" %}} - with: - push: true - tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest -``` -{{< /file >}} - -{{< /files >}} - -The workflow has two jobs: - -1. **test**: Builds the `test` stage of the Dockerfile and runs it. If tests fail, the workflow stops and `build_and_push` doesn't run. -2. **build_and_push**: Signs in to Docker Hub and the DHI registry, then builds and pushes the image. - -## Run the workflow - -Commit the changes and push them to the `main` branch. This workflow runs every time you push changes to `main`. You can find more information about workflow triggers in the [GitHub documentation](https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows). - -Go to the **Actions** tab of your GitHub repository. It displays the workflow. Selecting the workflow shows you the breakdown of all the steps. - -When the workflow is complete, go to your [repositories on Docker Hub](https://hub.docker.com/repositories). If you see the new repository in that list, the GitHub Actions workflow successfully pushed the image to Docker Hub. - -## Summary - -In this section, you learned how to set up a GitHub Actions workflow for your Node.js application that includes: - -- Running Vitest unit tests inside a container -- Building and pushing Docker images - -Related information: - -- [Introduction to GitHub Actions](/guides/gha.md) -- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) -- [docker/login-action](https://github.com/docker/login-action) -- [docker/build-push-action](https://github.com/docker/build-push-action) -- [Create a Docker Hub access token](/manuals/security/access-tokens.md#create-an-access-token) - -## Next steps - -In the next section, you'll learn how to inspect and generate supply chain -attestations for your image. See [Secure your supply chain](secure-supply-chain.md). diff --git a/content/guides/nodejs/containerize.md b/content/guides/nodejs/containerize.md deleted file mode 100644 index cc543e4f82d2..000000000000 --- a/content/guides/nodejs/containerize.md +++ /dev/null @@ -1,413 +0,0 @@ ---- -title: Containerize a Node.js application -linkTitle: Containerize your app -weight: 10 -keywords: node.js, node, containerize, initialize -description: Learn how to containerize a Node.js application with Docker. -aliases: - - /get-started/nodejs/build-images/ - - /language/nodejs/build-images/ - - /language/nodejs/run-containers/ - - /language/nodejs/containerize/ - - /guides/language/nodejs/containerize/ ---- - -## Prerequisites - -- You have installed the latest version of [Docker Desktop](/get-started/get-docker.md). -- You're familiar with basic Docker concepts. If you're new to Docker, start with [Get started](/get-started/introduction/). - -## Overview - -Containerizing your application means packaging it together with its -dependencies, configuration, and runtime into a single portable unit called a -container image. Running that image creates a container, an isolated process -that behaves the same on any machine, whether it's your laptop, a CI runner, or -a production server. - -In this section, you'll containerize a simple [Express.js](https://expressjs.com/) API written in TypeScript. You'll write a `Dockerfile` that describes how to build the image, add a `compose.yaml` file that defines how Docker runs your container, and then build and start the application with one command. - -You'll use [Docker Hardened Images](/dhi/) as the base. These are minimal, secure Node.js images maintained by Docker. - -This guide focuses on a backend Node.js API. If you're building a standalone frontend application, Docker has dedicated guides for [React.js](/guides/reactjs/), [Vue.js](/guides/vuejs/), [Angular](/guides/angular/), and [Next.js](/guides/nextjs/). - -## Create the application - -The sample application is a minimal Express API with a single endpoint that returns a JSON greeting. Create the following files in a new `nodejs-docker-example` directory. To create all the files at once, switch to the **Scaffold script** tab in the file browser and copy the shell command. - -{{< files name="nodejs-docker-example" >}} - -{{< file path="src/index.ts" status="new" >}} -```typescript -// A minimal Express application. -// The root endpoint (GET /) returns a JSON greeting. -// See https://expressjs.com/ for the framework reference. - -import express, { type Request, type Response } from 'express'; - -const app = express(); -const port = parseInt(process.env.PORT ?? '3000', 10); - -app.get('/', (_req: Request, res: Response) => { - res.json({ message: 'Hello World' }); -}); - -app.listen(port, () => { - console.log(`Server listening on port ${port}`); -}); -``` -{{< /file >}} - -{{< file path="package.json" status="new" >}} -```json -{ - "name": "nodejs-docker-example", - "version": "1.0.0", - "description": "A minimal Node.js TypeScript application.", - "main": "dist/index.js", - "scripts": { - "build": "tsc", - "start": "node dist/index.js", - "dev": "tsx watch src/index.ts" - }, - "dependencies": { - "express": "^4.21.2" - }, - "devDependencies": { - "@types/express": "^4.17.21", - "@types/node": "^22.0.0", - "tsx": "^4.19.3", - "typescript": "^5.8.3" - } -} -``` -{{< /file >}} - -{{< file path="tsconfig.json" status="new" >}} -```json -{ - // TypeScript compiler configuration for the Node.js application. - // Compiles src/ to dist/ as CommonJS modules targeting ES2022. - // See https://www.typescriptlang.org/tsconfig/ for all options. - "compilerOptions": { - "target": "ES2022", - "module": "commonjs", - "lib": ["ES2022"], - "outDir": "./dist", - "rootDir": "./src", - "strict": true, - "esModuleInterop": true, - "skipLibCheck": true - }, - "include": ["src/**/*"], - "exclude": ["node_modules", "dist"] -} -``` -{{< /file >}} - -{{< file path=".gitignore" status="new" >}} -```text -# Files and directories that Git should ignore. Covers Node.js dependencies, -# TypeScript build output, environment files, and common editor artifacts. -# See https://git-scm.com/docs/gitignore for syntax reference. - -node_modules/ -dist/ -.env -*.log -.DS_Store -coverage/ -db/password.txt -``` -{{< /file >}} - -{{< /files >}} - -If you have Node.js installed and want to verify the app works before containerizing it, you can run it locally. - -To run in development mode with hot-reload: - -```console -$ npm install -$ npm run dev -``` - -To run the compiled production build (matching what the Dockerfile does): - -```console -$ npm install -$ npm run build -$ npm start -``` - -Then open [http://localhost:3000](http://localhost:3000) in your browser. You should see `{"message":"Hello World"}`. - -If you don't have Node.js installed, skip ahead. The remaining steps run the application in a container, with no local Node.js required. - -## Create the Docker assets - -Sign in to the DHI registry so Docker can pull the Node.js base images during the build. The available Node.js images are listed in the [catalog](https://hub.docker.com/hardened-images/catalog/dhi/node). - -```console -$ docker login dhi.io -``` - -Add the following three files to your `nodejs-docker-example` directory. The `Dockerfile` describes how to build the image, `compose.yaml` defines how Docker runs the container, and `.dockerignore` keeps unwanted files out of the build context. - -> [!TIP] -> -> [Gordon](/ai/gordon/), Docker's AI assistant, can generate Docker assets for -> your project. Ask Gordon to create a Dockerfile, Compose file, and -> `.dockerignore` tailored to your application. - -{{< files name="nodejs-docker-example" >}} - -{{< file path="src/index.ts" >}} -```typescript -// A minimal Express application. -// The root endpoint (GET /) returns a JSON greeting. -// See https://expressjs.com/ for the framework reference. - -import express, { type Request, type Response } from 'express'; - -const app = express(); -const port = parseInt(process.env.PORT ?? '3000', 10); - -app.get('/', (_req: Request, res: Response) => { - res.json({ message: 'Hello World' }); -}); - -app.listen(port, () => { - console.log(`Server listening on port ${port}`); -}); -``` -{{< /file >}} - -{{< file path="package.json" >}} -```json -{ - "name": "nodejs-docker-example", - "version": "1.0.0", - "description": "A minimal Node.js TypeScript application.", - "main": "dist/index.js", - "scripts": { - "build": "tsc", - "start": "node dist/index.js", - "dev": "tsx watch src/index.ts" - }, - "dependencies": { - "express": "^4.21.2" - }, - "devDependencies": { - "@types/express": "^4.17.21", - "@types/node": "^22.0.0", - "tsx": "^4.19.3", - "typescript": "^5.8.3" - } -} -``` -{{< /file >}} - -{{< file path="tsconfig.json" >}} -```json -{ - // TypeScript compiler configuration for the Node.js application. - // Compiles src/ to dist/ as CommonJS modules targeting ES2022. - // See https://www.typescriptlang.org/tsconfig/ for all options. - "compilerOptions": { - "target": "ES2022", - "module": "commonjs", - "lib": ["ES2022"], - "outDir": "./dist", - "rootDir": "./src", - "strict": true, - "esModuleInterop": true, - "skipLibCheck": true - }, - "include": ["src/**/*"], - "exclude": ["node_modules", "dist"] -} -``` -{{< /file >}} - -{{< file path="Dockerfile" status="new" >}} -```dockerfile -# syntax=docker/dockerfile:1 - -# Comments are provided throughout this file to help you get started. -# If you need more help, visit the Dockerfile reference guide at -# https://docs.docker.com/go/dockerfile-reference/ - -# This Dockerfile uses Docker Hardened Images (DHI) for enhanced security. -# For more information, see https://docs.docker.com/dhi/ - -# Builder stage: install all dependencies and compile TypeScript. -FROM dhi.io/node:24-alpine3.23-dev AS builder - -WORKDIR /app - -# Install dependencies as a separate step to take advantage of Docker's -# caching. Leverage a cache mount to /root/.npm to speed up subsequent -# builds. Leverage a bind mount to package.json to avoid having to copy -# it into this layer. -RUN --mount=type=cache,target=/root/.npm \ - --mount=type=bind,source=package.json,target=package.json \ - npm install -# Once you create a package-lock.json by running npm install locally, switch to npm ci and bind both files: -# RUN --mount=type=cache,target=/root/.npm \ -# --mount=type=bind,source=package.json,target=package.json \ -# --mount=type=bind,source=package-lock.json,target=package-lock.json \ -# npm ci - -# Copy the source code into the container and compile TypeScript. -COPY . . -RUN npm run build - - -# Deps stage: install production dependencies only. -FROM dhi.io/node:24-alpine3.23-dev AS deps - -WORKDIR /app - -RUN --mount=type=cache,target=/root/.npm \ - --mount=type=bind,source=package.json,target=package.json \ - npm install --omit=dev -# Once you create a package-lock.json by running npm install locally, switch to npm ci and bind both files: -# RUN --mount=type=cache,target=/root/.npm \ -# --mount=type=bind,source=package.json,target=package.json \ -# --mount=type=bind,source=package-lock.json,target=package-lock.json \ -# npm ci --omit=dev - - -# Runner stage: minimal runtime image with compiled app and production deps. -FROM dhi.io/node:24-alpine3.23 AS runner - -ENV PATH=/app/node_modules/.bin:$PATH - -WORKDIR /app - -COPY --from=deps --chown=node:node /app/node_modules ./node_modules -COPY --from=builder --chown=node:node /app/dist ./dist - -# Expose the port that the application listens on. -EXPOSE 3000 - -# Run the application. -CMD ["node", "dist/index.js"] -``` -{{< /file >}} - -{{< file path="compose.yaml" status="new" >}} -```yaml -# Comments are provided throughout this file to help you get started. -# If you need more help, visit the Docker Compose reference guide at -# https://docs.docker.com/go/compose-spec-reference/ - -# Here the instructions define your application as a service called "server". -# This service is built from the Dockerfile in the current directory. -# You can add other services your application may depend on here, such as a -# database or a cache. For examples, see the Awesome Compose repository: -# https://github.com/docker/awesome-compose -services: - server: - build: - context: . - ports: - - 3000:3000 -``` -{{< /file >}} - -{{< file path=".dockerignore" status="new" >}} -```text -# Include any files or directories that you don't want to be copied to your -# container here (e.g., local build artifacts, temporary files, etc.). -# -# For more help, visit the .dockerignore file reference guide at -# https://docs.docker.com/go/build-context-dockerignore/ - -node_modules/ -dist/ -.env -.git -.gitignore -.DS_Store -npm-debug.log* -coverage/ -db/ -``` -{{< /file >}} - -{{< file path=".gitignore" >}} -```text -# Files and directories that Git should ignore. Covers Node.js dependencies, -# TypeScript build output, environment files, and common editor artifacts. -# See https://git-scm.com/docs/gitignore for syntax reference. - -node_modules/ -dist/ -.env -*.log -.DS_Store -coverage/ -db/password.txt -``` -{{< /file >}} - -{{< /files >}} - -The `Dockerfile` uses three stages. The `builder` stage installs all dependencies and compiles TypeScript. The `deps` stage does a fresh install of production-only dependencies. The `runner` stage copies the compiled output and production node_modules into a minimal runtime image that contains only Node.js. - -To learn more about each file, see the following: - -- [Dockerfile](/reference/dockerfile.md) -- [.dockerignore](/reference/dockerfile.md#dockerignore-file) -- [compose.yaml](/reference/compose-file/_index.md) - -## Run the application - -Inside the `nodejs-docker-example` directory, run the following command in a -terminal. - -```console -$ docker compose up --build -``` - -Open a browser and view the application at [http://localhost:3000](http://localhost:3000). You should see `{"message":"Hello World"}`. - -In the terminal, press `ctrl`+`c` to stop the application. - -### Run the application in the background - -You can run the application detached from the terminal by adding the `-d` -option. Inside the `nodejs-docker-example` directory, run the following command -in a terminal. - -```console -$ docker compose up --build -d -``` - -Open a browser and view the application at [http://localhost:3000](http://localhost:3000). - -In the terminal, run the following command to stop the application. - -```console -$ docker compose down -``` - -For more information about Compose commands, see the [Compose CLI -reference](/reference/cli/docker/compose/). - -## Summary - -In this section, you learned how to containerize and run a Node.js application using Docker. - -Related information: - -- [Docker Hardened Images](/dhi/) -- [Dockerfile reference](/reference/dockerfile.md) -- [Multi-stage builds](/manuals/build/building/multi-stage.md) -- [Docker Compose overview](/manuals/compose/_index.md) - -## Next steps - -In the next section, you'll take a look at how to set up a local development environment using Docker containers. diff --git a/content/guides/nodejs/deploy.md b/content/guides/nodejs/deploy.md deleted file mode 100644 index bbcdfeb1d429..000000000000 --- a/content/guides/nodejs/deploy.md +++ /dev/null @@ -1,274 +0,0 @@ ---- -title: Deploy your Node.js application -linkTitle: Deploy your app -weight: 50 -keywords: deploy, kubernetes, node, node.js, production -description: Learn how to deploy your containerized Node.js application to Kubernetes. -aliases: - - /language/nodejs/deploy/ - - /guides/language/nodejs/deploy/ ---- - -## Prerequisites - -- Complete all the previous sections of this guide, starting with [Containerize a Node.js application](containerize.md). -- [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. - -## Overview - -In this section, you'll deploy your containerized Node.js application to a local Kubernetes cluster using Docker Desktop. You'll create a Kubernetes manifest that describes how the application should run, including the application deployment, the PostgreSQL database, and the services that connect them. - -## Create a Kubernetes manifest - -Create a new file called `nodejs-docker-example-kubernetes.yaml` in your project root: - -{{< files name="nodejs-docker-example" >}} - -{{< file path="nodejs-docker-example-kubernetes.yaml" status="new" >}} -```yaml -apiVersion: v1 -kind: Namespace -metadata: - name: nodejs-docker-example - ---- -apiVersion: v1 -kind: ConfigMap -metadata: - name: app-config - namespace: nodejs-docker-example -data: - POSTGRES_SERVER: 'postgres' - POSTGRES_DB: 'example' - POSTGRES_USER: 'postgres' - ---- -apiVersion: v1 -kind: PersistentVolumeClaim -metadata: - name: postgres-pvc - namespace: nodejs-docker-example -spec: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 1Gi - ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - name: postgres - namespace: nodejs-docker-example -spec: - replicas: 1 - selector: - matchLabels: - app: postgres - template: - metadata: - labels: - app: postgres - spec: - containers: - - name: postgres - image: dhi.io/postgres:18 - ports: - - containerPort: 5432 - env: - - name: POSTGRES_DB - valueFrom: - configMapKeyRef: - name: app-config - key: POSTGRES_DB - - name: POSTGRES_USER - valueFrom: - configMapKeyRef: - name: app-config - key: POSTGRES_USER - - name: POSTGRES_PASSWORD - valueFrom: - secretKeyRef: - name: app-secrets - key: db-password - volumeMounts: - - name: postgres-storage - mountPath: /var/lib/postgresql - readinessProbe: - exec: - command: [pg_isready] - initialDelaySeconds: 5 - periodSeconds: 5 - volumes: - - name: postgres-storage - persistentVolumeClaim: - claimName: postgres-pvc - ---- -apiVersion: v1 -kind: Service -metadata: - name: postgres - namespace: nodejs-docker-example -spec: - type: ClusterIP - ports: - - port: 5432 - targetPort: 5432 - selector: - app: postgres - ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - name: server - namespace: nodejs-docker-example -spec: - replicas: 2 - selector: - matchLabels: - app: server - template: - metadata: - labels: - app: server - spec: - containers: - - name: server - image: DOCKER_USERNAME/nodejs-docker-example:latest - ports: - - containerPort: 3000 - env: - - name: POSTGRES_SERVER - valueFrom: - configMapKeyRef: - name: app-config - key: POSTGRES_SERVER - - name: POSTGRES_DB - valueFrom: - configMapKeyRef: - name: app-config - key: POSTGRES_DB - - name: POSTGRES_USER - valueFrom: - configMapKeyRef: - name: app-config - key: POSTGRES_USER - - name: POSTGRES_PASSWORD - valueFrom: - secretKeyRef: - name: app-secrets - key: db-password - readinessProbe: - httpGet: - path: /health - port: 3000 - initialDelaySeconds: 10 - periodSeconds: 5 - ---- -apiVersion: v1 -kind: Service -metadata: - name: server - namespace: nodejs-docker-example -spec: - type: ClusterIP - ports: - - port: 3000 - targetPort: 3000 - selector: - app: server -``` -{{< /file >}} - -{{< /files >}} - -Before applying the manifest, replace `DOCKER_USERNAME` in the `server` deployment's image field with your Docker Hub username. - -## Deploy to Kubernetes - -Apply the manifest to your local Kubernetes cluster: - -```console -$ kubectl apply -f nodejs-docker-example-kubernetes.yaml -``` - -You should see output confirming that the resources were created: - -```console -namespace/nodejs-docker-example created -configmap/app-config created -persistentvolumeclaim/postgres-pvc created -deployment.apps/postgres created -service/postgres created -deployment.apps/server created -service/server created -``` - -Then create the database secret from your password file: - -```console -$ kubectl create secret generic app-secrets \ - --namespace nodejs-docker-example \ - --from-file=db-password=db/password.txt -``` - -## Verify the deployment - -Check that your pods are running: - -```console -$ kubectl get pods -n nodejs-docker-example -``` - -Wait until all pods show `Running` in the STATUS column. Then verify your services: - -```console -$ kubectl get services -n nodejs-docker-example -``` - -## Access the application - -Use port forwarding to access the application from your local machine: - -```console -$ kubectl port-forward -n nodejs-docker-example service/server 3000:3000 -``` - -Open a new terminal and make a request to the application: - -```console -$ curl http://localhost:3000 -{"message":"Hello World"} -``` - -You can also create a hero: - -```console -$ curl -X POST http://localhost:3000/heroes/ \ - -H 'Content-Type: application/json' \ - -d '{"name": "my hero", "secret_name": "austing", "age": 12}' -``` - -## Clean up - -When you're done testing, remove the deployment: - -```console -$ kubectl delete -f nodejs-docker-example-kubernetes.yaml -``` - -## Summary - -In this section, you deployed your containerized Node.js application to Kubernetes. You created a manifest that defines the application and database deployments, applied it to a local cluster, and verified the application is accessible. - -Related information: - -- [Kubernetes documentation](https://kubernetes.io/docs/home/) -- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) -- [`kubectl` CLI reference](https://kubernetes.io/docs/reference/kubectl/) -- [Kubernetes Deployment resource](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) -- [Kubernetes Service resource](https://kubernetes.io/docs/concepts/services-networking/service/) diff --git a/content/guides/nodejs/develop.md b/content/guides/nodejs/develop.md deleted file mode 100644 index 68a8d0fc7f79..000000000000 --- a/content/guides/nodejs/develop.md +++ /dev/null @@ -1,633 +0,0 @@ ---- -title: Use containers for Node.js development -linkTitle: Develop your app -weight: 20 -keywords: node, node.js, development -description: Learn how to develop your Node.js application locally using containers. -aliases: - - /get-started/nodejs/develop/ - - /language/nodejs/develop/ - - /guides/language/nodejs/develop/ ---- - -## Prerequisites - -Complete [Containerize a Node.js application](containerize.md). - -## Overview - -Once your application runs in a container, the next step is making the container part of your everyday development workflow. Code changes should show up quickly, and services your app depends on, like databases, should run right alongside it. - -In this section, you'll adapt the Dockerfile for local development by renaming the `builder` stage to `dev` and pointing Compose at it. You'll also update the application to connect to a PostgreSQL database, add a database service to `compose.yaml`, persist data in a named volume, enable Compose Watch so changes in your editor are picked up without a manual rebuild, and set up Node.js debugging so you can attach VS Code or Chrome DevTools to the running container. - -## Update the application - -You'll update your application to connect to a PostgreSQL database. Continue working in your `nodejs-docker-example` directory. - -Replace `src/index.ts` and `package.json` with the following contents. The file browser shows only the files that change in this step. - -> [!NOTE] -> -> The application won't run yet after this step. It tries to connect to a -> PostgreSQL database that doesn't exist. The next two sections add the -> database service and the Docker configuration needed to run everything -> together. - -{{< files name="nodejs-docker-example" >}} - -{{< file path="src/index.ts" status="modified" >}} -```typescript -// Express application backed by a PostgreSQL database. -// Creates a heroes table at startup. -// Endpoints: GET / (greeting), GET /health (health check), POST /heroes/ (create), GET /heroes/ (list). -// See https://expressjs.com/ and https://node-postgres.com/ - -import express, { type Request, type Response } from 'express'; -import { Pool } from 'pg'; -import { readFileSync } from 'fs'; - -const app = express(); -const port = parseInt(process.env.PORT ?? '3000', 10); - -app.use(express.json()); - -function getPassword(): string { - const passwordFile = process.env.POSTGRES_PASSWORD_FILE; - if (passwordFile) { - return readFileSync(passwordFile, 'utf8').trim(); - } - return process.env.POSTGRES_PASSWORD ?? ''; -} - -const pool = new Pool({ - host: process.env.POSTGRES_SERVER, - port: 5432, - database: process.env.POSTGRES_DB, - user: process.env.POSTGRES_USER, - password: getPassword(), -}); - -pool - .query( - `CREATE TABLE IF NOT EXISTS heroes ( - id SERIAL PRIMARY KEY, - name TEXT NOT NULL, - secret_name TEXT NOT NULL, - age INTEGER - )`, - ) - .catch(console.error); - -app.get('/', (_req: Request, res: Response) => { - res.json({ message: 'Hello World' }); -}); - -app.get('/health', (_req: Request, res: Response) => { - res.json({ status: 'ok' }); -}); - -app.post('/heroes/', async (req: Request, res: Response) => { - const { name, secret_name, age } = req.body as { - name: string; - secret_name: string; - age?: number; - }; - const result = await pool.query( - 'INSERT INTO heroes (name, secret_name, age) VALUES ($1, $2, $3) RETURNING *', - [name, secret_name, age], - ); - res.json(result.rows[0]); -}); - -app.get('/heroes/', async (_req: Request, res: Response) => { - const result = await pool.query('SELECT * FROM heroes'); - res.json(result.rows); -}); - -app.listen(port, () => { - console.log(`Server listening on port ${port}`); -}); -``` -{{< /file >}} - -{{< file path="package.json" status="modified" hl_lines="13,18" >}} -```json -{ - "name": "nodejs-docker-example", - "version": "1.0.0", - "description": "A minimal Node.js TypeScript application.", - "main": "dist/index.js", - "scripts": { - "build": "tsc", - "start": "node dist/index.js", - "dev": "tsx watch src/index.ts" - }, - "dependencies": { - "express": "^4.21.2", - "pg": "^8.16.0" - }, - "devDependencies": { - "@types/express": "^4.17.21", - "@types/node": "^22.0.0", - "@types/pg": "^8.11.0", - "tsx": "^4.19.3", - "typescript": "^5.8.3" - } -} -``` -{{< /file >}} - -{{< /files >}} - -## Update Docker assets - -Replace `Dockerfile` and `compose.yaml` with the following. - -{{< files name="nodejs-docker-example" >}} - -{{< file path="Dockerfile" status="modified" hl_lines="12,34,37,63" >}} -```dockerfile -# syntax=docker/dockerfile:1 - -# Comments are provided throughout this file to help you get started. -# If you need more help, visit the Dockerfile reference guide at -# https://docs.docker.com/go/dockerfile-reference/ - -# This Dockerfile uses Docker Hardened Images (DHI) for enhanced security. -# For more information, see https://docs.docker.com/dhi/ - -# Development stage: install all dependencies, compile TypeScript, and -# serve with hot-reload. Used directly in development via compose.yaml. -FROM dhi.io/node:24-alpine3.23-dev AS dev - -WORKDIR /app - -# Install dependencies as a separate step to take advantage of Docker's -# caching. Leverage a cache mount to /root/.npm to speed up subsequent -# builds. Leverage a bind mount to package.json to avoid having to copy -# it into this layer. -RUN --mount=type=cache,target=/root/.npm \ - --mount=type=bind,source=package.json,target=package.json \ - npm install -# Once you create a package-lock.json by running npm install locally, switch to npm ci and bind both files: -# RUN --mount=type=cache,target=/root/.npm \ -# --mount=type=bind,source=package.json,target=package.json \ -# --mount=type=bind,source=package-lock.json,target=package-lock.json \ -# npm ci - -# Copy the source code into the container and compile TypeScript. -COPY . . -RUN npm run build - -# Expose the port that the application listens on. -EXPOSE 3000 - -# Run the application in development mode. -CMD ["npm", "run", "dev"] - - -# Deps stage: install production dependencies only. -FROM dhi.io/node:24-alpine3.23-dev AS deps - -WORKDIR /app - -RUN --mount=type=cache,target=/root/.npm \ - --mount=type=bind,source=package.json,target=package.json \ - npm install --omit=dev -# Once you create a package-lock.json by running npm install locally, switch to npm ci and bind both files: -# RUN --mount=type=cache,target=/root/.npm \ -# --mount=type=bind,source=package.json,target=package.json \ -# --mount=type=bind,source=package-lock.json,target=package-lock.json \ -# npm ci --omit=dev - - -# Runner stage: minimal runtime image with compiled app and production deps. -FROM dhi.io/node:24-alpine3.23 AS runner - -ENV PATH=/app/node_modules/.bin:$PATH - -WORKDIR /app - -COPY --from=deps --chown=node:node /app/node_modules ./node_modules -COPY --from=dev --chown=node:node /app/dist ./dist - -# Expose the port that the application listens on. -EXPOSE 3000 - -# Run the application. -CMD ["node", "dist/index.js"] -``` -{{< /file >}} - -{{< file path="compose.yaml" status="modified" hl_lines="8" >}} -```yaml -services: - # Application service. The `target: dev` line builds the development - # image (includes tsx and dev tooling); the runner stage of the - # Dockerfile is unused in development. - server: - build: - context: . - target: dev - ports: - - 3000:3000 -``` -{{< /file >}} - -{{< /files >}} - -### About these changes - -The `builder` stage from containerize is renamed to `dev` and gains `EXPOSE 3000` and `CMD ["npm", "run", "dev"]`, which runs `tsx watch` for hot-reload. The `deps` and `runner` stages are unchanged. - -In `compose.yaml`, the new `target: dev` line tells Compose to build and run the `dev` stage during development. Unlike the production image, the development image includes `tsx` and other dev tooling. If you need a shell in a running production container, use [Docker Debug](/reference/cli/docker/debug/) instead. - -The build step runs `tsc`, which compiles each TypeScript file into a corresponding JavaScript file. [esbuild](https://esbuild.github.io/) is a popular alternative that bundles everything into a single output file and builds significantly faster. To switch, replace the `tsc` call in `package.json` with an esbuild command and update the `COPY --from=dev` path in the `runner` stage to match esbuild's output. - -## Add a local database and persist data - -You can use containers to set up local services, like a database. In this section, you'll update the `compose.yaml` file to define a database service and a volume to persist data, and add a `db/password.txt` file that holds the database password. - -{{< files name="nodejs-docker-example" >}} - -{{< file path="compose.yaml" status="modified" hl_lines="11-46" >}} -```yaml -services: - # Application service. The `target: dev` line builds the development - # image (includes tsx and dev tooling); the runner stage of the - # Dockerfile is unused in development. - server: - build: - context: . - target: dev - ports: - - 3000:3000 - environment: - - POSTGRES_SERVER=db - - POSTGRES_USER=postgres - - POSTGRES_DB=example - - POSTGRES_PASSWORD_FILE=/run/secrets/db-password - depends_on: - db: - condition: service_healthy - secrets: - - db-password - # Database service. Reads the password from a Docker secret mounted at - # /run/secrets/db-password. Compose waits for the healthcheck to pass - # before starting the server, via the server's depends_on. - db: - image: dhi.io/postgres:18 - restart: always - user: postgres - secrets: - - db-password - volumes: - - db-data:/var/lib/postgresql - environment: - - POSTGRES_DB=example - - POSTGRES_PASSWORD_FILE=/run/secrets/db-password - expose: - - 5432 - healthcheck: - test: ["CMD", "pg_isready"] - interval: 10s - timeout: 5s - retries: 5 -volumes: - db-data: -secrets: - db-password: - file: db/password.txt -``` -{{< /file >}} - -{{< file path="db/password.txt" status="new" >}} -```text -mysecretpassword -``` -{{< /file >}} - -{{< /files >}} - -> [!NOTE] -> -> To learn more about the instructions in the Compose file, see [Compose file -> reference](/reference/compose-file/). - -Now, run the following `docker compose up` command to start your application. - -```console -$ docker compose up --build -``` - -Now test your API endpoint. Open a new terminal and make a request to the server using the curl commands. - -Create an object with a POST request: - -```console -$ curl -X 'POST' \ - 'http://localhost:3000/heroes/' \ - -H 'accept: application/json' \ - -H 'Content-Type: application/json' \ - -d '{ - "name": "my hero", - "secret_name": "austing", - "age": 12 -}' -``` - -You should receive the following response: - -```json -{ - "id": 1, - "name": "my hero", - "secret_name": "austing", - "age": 12 -} -``` - -Now make a GET request: - -```console -$ curl http://localhost:3000/heroes/ -``` - -You should receive the same response because it's the only object in the database. - -Press `ctrl`+`c` in the terminal to stop your application. - -## Automatically update services - -Use Compose Watch to automatically update your running Compose services as you edit and save your code. For more details about Compose Watch, see [Use Compose Watch](/manuals/compose/how-tos/file-watch.md). - -Open your `compose.yaml` file in an IDE or text editor and add the highlighted Compose Watch instructions. - -{{< files name="nodejs-docker-example" >}} - -{{< file path="compose.yaml" status="modified" hl_lines="21-27" >}} -```yaml -services: - # Application service. The `target: dev` line builds the development - # image (includes tsx and dev tooling); the runner stage of the - # Dockerfile is unused in development. - server: - build: - context: . - target: dev - ports: - - 3000:3000 - environment: - - POSTGRES_SERVER=db - - POSTGRES_USER=postgres - - POSTGRES_DB=example - - POSTGRES_PASSWORD_FILE=/run/secrets/db-password - depends_on: - db: - condition: service_healthy - secrets: - - db-password - develop: - watch: - - action: sync - path: ./src - target: /app/src - - action: rebuild - path: package.json - db: - image: dhi.io/postgres:18 - restart: always - user: postgres - secrets: - - db-password - volumes: - - db-data:/var/lib/postgresql - environment: - - POSTGRES_DB=example - - POSTGRES_PASSWORD_FILE=/run/secrets/db-password - expose: - - 5432 - healthcheck: - test: ["CMD", "pg_isready"] - interval: 10s - timeout: 5s - retries: 5 -volumes: - db-data: -secrets: - db-password: - file: db/password.txt -``` -{{< /file >}} - -{{< /files >}} - -Run the following command to run your application with Compose Watch. - -```console -$ docker compose watch -``` - -In a terminal, curl the application to get a response. - -```console -$ curl http://localhost:3000 -{"message":"Hello World"} -``` - -Any changes to the application's source files on your local machine will now be immediately reflected in the running container. - -Open `nodejs-docker-example/src/index.ts` in an IDE or text editor and update the `Hello World` string by adding a few more exclamation marks. - -```diff -- res.json({ message: 'Hello World' }); -+ res.json({ message: 'Hello World!!!' }); -``` - -Save the changes to `src/index.ts` and then wait a few seconds for the application to reload. Curl the application again and verify that the updated text appears. - -```console -$ curl http://localhost:3000 -{"message":"Hello World!!!"} -``` - -Press `ctrl`+`c` in the terminal to stop your application. - -## Debug your application - -`tsx watch` supports the Node.js inspector protocol, so you can attach a debugger from VS Code or Chrome DevTools and set breakpoints directly in your TypeScript source files. - -Update the `dev` script in `package.json` to start the inspector. The `--inspect=0.0.0.0:9229` flag tells Node.js to listen for debugger connections on all network interfaces at port 9229. Using `0.0.0.0` rather than `localhost` is necessary so the debugger is reachable from outside the container. Also expose the debug port in `compose.yaml`, and add a `.vscode/launch.json` file that tells VS Code how to attach to the running inspector. - -{{< files name="nodejs-docker-example" >}} - -{{< file path="package.json" status="modified" hl_lines="9" >}} -```json -{ - "name": "nodejs-docker-example", - "version": "1.0.0", - "description": "A minimal Node.js TypeScript application.", - "main": "dist/index.js", - "scripts": { - "build": "tsc", - "start": "node dist/index.js", - "dev": "tsx watch --inspect=0.0.0.0:9229 src/index.ts" - }, - "dependencies": { - "express": "^4.21.2", - "pg": "^8.16.0" - }, - "devDependencies": { - "@types/express": "^4.17.21", - "@types/node": "^22.0.0", - "@types/pg": "^8.11.0", - "tsx": "^4.19.3", - "typescript": "^5.8.3" - } -} -``` -{{< /file >}} - -{{< file path=".vscode/launch.json" status="new" >}} -```json -{ - "version": "0.2.0", - "configurations": [ - { - "name": "Attach to Docker Container", - "type": "node", - "request": "attach", - "port": 9229, - "address": "localhost", - "localRoot": "${workspaceFolder}", - "remoteRoot": "/app", - "protocol": "inspector", - "restart": true, - "sourceMaps": true, - "skipFiles": ["/**"] - } - ] -} -``` -{{< /file >}} - -{{< file path="compose.yaml" status="modified" hl_lines="11" >}} -```yaml -services: - # Application service. The `target: dev` line builds the development - # image (includes tsx and dev tooling); the runner stage of the - # Dockerfile is unused in development. - server: - build: - context: . - target: dev - ports: - - 3000:3000 - - 9229:9229 - environment: - - POSTGRES_SERVER=db - - POSTGRES_USER=postgres - - POSTGRES_DB=example - - POSTGRES_PASSWORD_FILE=/run/secrets/db-password - depends_on: - db: - condition: service_healthy - secrets: - - db-password - develop: - watch: - - action: sync - path: ./src - target: /app/src - - action: rebuild - path: package.json - db: - image: dhi.io/postgres:18 - restart: always - user: postgres - secrets: - - db-password - volumes: - - db-data:/var/lib/postgresql - environment: - - POSTGRES_DB=example - - POSTGRES_PASSWORD_FILE=/run/secrets/db-password - expose: - - 5432 - healthcheck: - test: ["CMD", "pg_isready"] - interval: 10s - timeout: 5s - retries: 5 -volumes: - db-data: -secrets: - db-password: - file: db/password.txt -``` -{{< /file >}} - -{{< /files >}} - -Rebuild and restart with the updated configuration: - -```console -$ docker compose up --build -``` - -When the inspector is ready, you'll see a line like the following in the logs: - -```text -Debugger listening on ws://0.0.0.0:9229/... -``` - -### VS Code - -With `.vscode/launch.json` in place, attach the debugger using the Debug panel. - -Open the Debug panel (`Ctrl+Shift+D` on Windows and Linux, `Cmd+Shift+D` on Mac), select **Attach to Docker Container**, and press `F5`. You can now set breakpoints in your TypeScript source files under `src/`. - -### Chrome DevTools - -You can also use the built-in Node.js inspector in Chrome without any editor setup. - -1. Open Chrome and go to `chrome://inspect`. - -2. Select **Configure** and add `localhost:9229`. - -3. When your Node.js target appears in the list, select **inspect**. - -### Troubleshoot the debugger - -If the debugger doesn't connect, verify the container is running and the port is mapped correctly: - -```console -$ docker compose ps -$ docker compose logs server -``` - -The logs should include a line like: - -```text -Debugger listening on ws://0.0.0.0:9229/... -``` - -If that line is missing, confirm the `dev` script in `package.json` includes `--inspect=0.0.0.0:9229` and that `9229:9229` appears in the `ports` list for the `server` service in `compose.yaml`. - -For more details about Node.js debugging, see the [Node.js debugging guide](https://nodejs.org/en/docs/guides/debugging-getting-started). - -## Summary - -In this section, you set up a Compose file with a local database and persistent storage, set up Compose Watch to automatically sync code changes, and configured a debugger that attaches from VS Code and Chrome DevTools. - -Related information: - -- [Compose file reference](/reference/compose-file/) -- [Compose secrets](/reference/compose-file/secrets.md) -- [Compose Watch](/manuals/compose/how-tos/file-watch.md) -- [Multi-stage builds](/manuals/build/building/multi-stage.md) -- [Node.js debugging guide](https://nodejs.org/en/docs/guides/debugging-getting-started) - -## Next steps - -In the next section, you'll learn how to run tests using Docker. diff --git a/content/guides/nodejs/index.md b/content/guides/nodejs/index.md new file mode 100644 index 000000000000..ca65a17aaddc --- /dev/null +++ b/content/guides/nodejs/index.md @@ -0,0 +1,1868 @@ +--- +title: Node.js language-specific guide +linkTitle: Node.js +description: Containerize and develop Node.js applications using Docker +keywords: getting started, node, node.js +summary: | + This guide explains how to containerize Node.js applications using Docker. +aliases: + - /guides/nodejs/containerize/ + - /guides/nodejs/develop/ + - /guides/nodejs/run-tests/ + - /guides/nodejs/configure-github-actions/ + - /guides/nodejs/deploy/ + - /guides/nodejs/secure-supply-chain/ + - /language/nodejs/ + - /guides/language/nodejs/ + - /get-started/nodejs/build-images/ + - /language/nodejs/build-images/ + - /language/nodejs/run-containers/ + - /language/nodejs/containerize/ + - /get-started/nodejs/develop/ + - /language/nodejs/develop/ + - /language/nodejs/run-tests/ + - /language/nodejs/configure-ci-cd/ + - /language/nodejs/deploy/ +params: + tags: [languages] + time: 20 minutes +--- + + +[Node.js](https://nodejs.org/en) is a JavaScript runtime for building server-side applications. This guide shows you how to containerize a TypeScript Node.js application using Docker, starting from a simple Express API and progressively adding features like a database and CI/CD. + +This guide focuses on a backend Node.js API. If you're building a standalone frontend application, Docker has dedicated guides for [React.js](/guides/reactjs/), [Vue.js](/guides/vuejs/), [Angular](/guides/angular/), and [Next.js](/guides/nextjs/). + +> **Acknowledgment** +> +> Docker thanks [Kristiyan Velkov](https://www.linkedin.com/in/kristiyan-velkov-763130b3/) for his contribution to this guide. + +## What will you learn? + +In this guide, you'll learn how to: + +- Containerize and run a Node.js application using Docker. +- Set up a local development environment using containers. +- Run tests inside a Docker container. +- Configure GitHub Actions for CI/CD with Docker. +- Inspect and generate supply chain attestations for your image. +- Deploy your containerized Node.js application to Kubernetes. + +Start by containerizing a Node.js application. + +## Prerequisites + +- Basic understanding of [JavaScript](https://developer.mozilla.org/en-US/docs/Web/JavaScript) and [TypeScript](https://www.typescriptlang.org/). +- Basic knowledge of [Node.js](https://nodejs.org/en) and [npm](https://docs.npmjs.com/about-npm). +- Familiarity with Docker concepts such as images, containers, and Dockerfiles. If you're new to Docker, start with the [Docker basics](/get-started/docker-concepts/the-basics/what-is-a-container.md) guide. + +## Containerize a Node.js application + +### Prerequisites + +- You have installed the latest version of [Docker Desktop](/get-started/get-docker.md). +- You're familiar with basic Docker concepts. If you're new to Docker, start with [Get started](/get-started/introduction/). + +### Overview + +Containerizing your application means packaging it together with its +dependencies, configuration, and runtime into a single portable unit called a +container image. Running that image creates a container, an isolated process +that behaves the same on any machine, whether it's your laptop, a CI runner, or +a production server. + +In this section, you'll containerize a simple [Express.js](https://expressjs.com/) API written in TypeScript. You'll write a `Dockerfile` that describes how to build the image, add a `compose.yaml` file that defines how Docker runs your container, and then build and start the application with one command. + +You'll use [Docker Hardened Images](/dhi/) as the base. These are minimal, secure Node.js images maintained by Docker. + +This guide focuses on a backend Node.js API. If you're building a standalone frontend application, Docker has dedicated guides for [React.js](/guides/reactjs/), [Vue.js](/guides/vuejs/), [Angular](/guides/angular/), and [Next.js](/guides/nextjs/). + +### Create the application + +The sample application is a minimal Express API with a single endpoint that returns a JSON greeting. Create the following files in a new `nodejs-docker-example` directory. To create all the files at once, switch to the **Scaffold script** tab in the file browser and copy the shell command. + +{{< files name="nodejs-docker-example" >}} + +{{< file path="src/index.ts" status="new" >}} +```typescript +// A minimal Express application. +// The root endpoint (GET /) returns a JSON greeting. +// See https://expressjs.com/ for the framework reference. + +import express, { type Request, type Response } from 'express'; + +const app = express(); +const port = parseInt(process.env.PORT ?? '3000', 10); + +app.get('/', (_req: Request, res: Response) => { + res.json({ message: 'Hello World' }); +}); + +app.listen(port, () => { + console.log(`Server listening on port ${port}`); +}); +``` +{{< /file >}} + +{{< file path="package.json" status="new" >}} +```json +{ + "name": "nodejs-docker-example", + "version": "1.0.0", + "description": "A minimal Node.js TypeScript application.", + "main": "dist/index.js", + "scripts": { + "build": "tsc", + "start": "node dist/index.js", + "dev": "tsx watch src/index.ts" + }, + "dependencies": { + "express": "^4.21.2" + }, + "devDependencies": { + "@types/express": "^4.17.21", + "@types/node": "^22.0.0", + "tsx": "^4.19.3", + "typescript": "^5.8.3" + } +} +``` +{{< /file >}} + +{{< file path="tsconfig.json" status="new" >}} +```json +{ + // TypeScript compiler configuration for the Node.js application. + // Compiles src/ to dist/ as CommonJS modules targeting ES2022. + // See https://www.typescriptlang.org/tsconfig/ for all options. + "compilerOptions": { + "target": "ES2022", + "module": "commonjs", + "lib": ["ES2022"], + "outDir": "./dist", + "rootDir": "./src", + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true + }, + "include": ["src/**/*"], + "exclude": ["node_modules", "dist"] +} +``` +{{< /file >}} + +{{< file path=".gitignore" status="new" >}} +```text +# Files and directories that Git should ignore. Covers Node.js dependencies, +# TypeScript build output, environment files, and common editor artifacts. +# See https://git-scm.com/docs/gitignore for syntax reference. + +node_modules/ +dist/ +.env +*.log +.DS_Store +coverage/ +db/password.txt +``` +{{< /file >}} + +{{< /files >}} + +If you have Node.js installed and want to verify the app works before containerizing it, you can run it locally. + +To run in development mode with hot-reload: + +```console +$ npm install +$ npm run dev +``` + +To run the compiled production build (matching what the Dockerfile does): + +```console +$ npm install +$ npm run build +$ npm start +``` + +Then open [http://localhost:3000](http://localhost:3000) in your browser. You should see `{"message":"Hello World"}`. + +If you don't have Node.js installed, skip ahead. The remaining steps run the application in a container, with no local Node.js required. + +### Create the Docker assets + +Sign in to the DHI registry so Docker can pull the Node.js base images during the build. The available Node.js images are listed in the [catalog](https://hub.docker.com/hardened-images/catalog/dhi/node). + +```console +$ docker login dhi.io +``` + +Add the following three files to your `nodejs-docker-example` directory. The `Dockerfile` describes how to build the image, `compose.yaml` defines how Docker runs the container, and `.dockerignore` keeps unwanted files out of the build context. + +> [!TIP] +> +> [Gordon](/ai/gordon/), Docker's AI assistant, can generate Docker assets for +> your project. Ask Gordon to create a Dockerfile, Compose file, and +> `.dockerignore` tailored to your application. + +{{< files name="nodejs-docker-example" >}} + +{{< file path="src/index.ts" >}} +```typescript +// A minimal Express application. +// The root endpoint (GET /) returns a JSON greeting. +// See https://expressjs.com/ for the framework reference. + +import express, { type Request, type Response } from 'express'; + +const app = express(); +const port = parseInt(process.env.PORT ?? '3000', 10); + +app.get('/', (_req: Request, res: Response) => { + res.json({ message: 'Hello World' }); +}); + +app.listen(port, () => { + console.log(`Server listening on port ${port}`); +}); +``` +{{< /file >}} + +{{< file path="package.json" >}} +```json +{ + "name": "nodejs-docker-example", + "version": "1.0.0", + "description": "A minimal Node.js TypeScript application.", + "main": "dist/index.js", + "scripts": { + "build": "tsc", + "start": "node dist/index.js", + "dev": "tsx watch src/index.ts" + }, + "dependencies": { + "express": "^4.21.2" + }, + "devDependencies": { + "@types/express": "^4.17.21", + "@types/node": "^22.0.0", + "tsx": "^4.19.3", + "typescript": "^5.8.3" + } +} +``` +{{< /file >}} + +{{< file path="tsconfig.json" >}} +```json +{ + // TypeScript compiler configuration for the Node.js application. + // Compiles src/ to dist/ as CommonJS modules targeting ES2022. + // See https://www.typescriptlang.org/tsconfig/ for all options. + "compilerOptions": { + "target": "ES2022", + "module": "commonjs", + "lib": ["ES2022"], + "outDir": "./dist", + "rootDir": "./src", + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true + }, + "include": ["src/**/*"], + "exclude": ["node_modules", "dist"] +} +``` +{{< /file >}} + +{{< file path="Dockerfile" status="new" >}} +```dockerfile +# syntax=docker/dockerfile:1 + +# Comments are provided throughout this file to help you get started. +# If you need more help, visit the Dockerfile reference guide at +# https://docs.docker.com/go/dockerfile-reference/ + +# This Dockerfile uses Docker Hardened Images (DHI) for enhanced security. +# For more information, see https://docs.docker.com/dhi/ + +# Builder stage: install all dependencies and compile TypeScript. +FROM dhi.io/node:24-alpine3.23-dev AS builder + +WORKDIR /app + +# Install dependencies as a separate step to take advantage of Docker's +# caching. Leverage a cache mount to /root/.npm to speed up subsequent +# builds. Leverage a bind mount to package.json to avoid having to copy +# it into this layer. +RUN --mount=type=cache,target=/root/.npm \ + --mount=type=bind,source=package.json,target=package.json \ + npm install +# Once you create a package-lock.json by running npm install locally, switch to npm ci and bind both files: +# RUN --mount=type=cache,target=/root/.npm \ +# --mount=type=bind,source=package.json,target=package.json \ +# --mount=type=bind,source=package-lock.json,target=package-lock.json \ +# npm ci + +# Copy the source code into the container and compile TypeScript. +COPY . . +RUN npm run build + + +# Deps stage: install production dependencies only. +FROM dhi.io/node:24-alpine3.23-dev AS deps + +WORKDIR /app + +RUN --mount=type=cache,target=/root/.npm \ + --mount=type=bind,source=package.json,target=package.json \ + npm install --omit=dev +# Once you create a package-lock.json by running npm install locally, switch to npm ci and bind both files: +# RUN --mount=type=cache,target=/root/.npm \ +# --mount=type=bind,source=package.json,target=package.json \ +# --mount=type=bind,source=package-lock.json,target=package-lock.json \ +# npm ci --omit=dev + + +# Runner stage: minimal runtime image with compiled app and production deps. +FROM dhi.io/node:24-alpine3.23 AS runner + +ENV PATH=/app/node_modules/.bin:$PATH + +WORKDIR /app + +COPY --from=deps --chown=node:node /app/node_modules ./node_modules +COPY --from=builder --chown=node:node /app/dist ./dist + +# Expose the port that the application listens on. +EXPOSE 3000 + +# Run the application. +CMD ["node", "dist/index.js"] +``` +{{< /file >}} + +{{< file path="compose.yaml" status="new" >}} +```yaml +# Comments are provided throughout this file to help you get started. +# If you need more help, visit the Docker Compose reference guide at +# https://docs.docker.com/go/compose-spec-reference/ + +# Here the instructions define your application as a service called "server". +# This service is built from the Dockerfile in the current directory. +# You can add other services your application may depend on here, such as a +# database or a cache. For examples, see the Awesome Compose repository: +# https://github.com/docker/awesome-compose +services: + server: + build: + context: . + ports: + - 3000:3000 +``` +{{< /file >}} + +{{< file path=".dockerignore" status="new" >}} +```text +# Include any files or directories that you don't want to be copied to your +# container here (e.g., local build artifacts, temporary files, etc.). +# +# For more help, visit the .dockerignore file reference guide at +# https://docs.docker.com/go/build-context-dockerignore/ + +node_modules/ +dist/ +.env +.git +.gitignore +.DS_Store +npm-debug.log* +coverage/ +db/ +``` +{{< /file >}} + +{{< file path=".gitignore" >}} +```text +# Files and directories that Git should ignore. Covers Node.js dependencies, +# TypeScript build output, environment files, and common editor artifacts. +# See https://git-scm.com/docs/gitignore for syntax reference. + +node_modules/ +dist/ +.env +*.log +.DS_Store +coverage/ +db/password.txt +``` +{{< /file >}} + +{{< /files >}} + +The `Dockerfile` uses three stages. The `builder` stage installs all dependencies and compiles TypeScript. The `deps` stage does a fresh install of production-only dependencies. The `runner` stage copies the compiled output and production node_modules into a minimal runtime image that contains only Node.js. + +To learn more about each file, see the following: + +- [Dockerfile](/reference/dockerfile.md) +- [.dockerignore](/reference/dockerfile.md#dockerignore-file) +- [compose.yaml](/reference/compose-file/_index.md) + +### Run the application + +Inside the `nodejs-docker-example` directory, run the following command in a +terminal. + +```console +$ docker compose up --build +``` + +Open a browser and view the application at [http://localhost:3000](http://localhost:3000). You should see `{"message":"Hello World"}`. + +In the terminal, press `ctrl`+`c` to stop the application. + +#### Run the application in the background + +You can run the application detached from the terminal by adding the `-d` +option. Inside the `nodejs-docker-example` directory, run the following command +in a terminal. + +```console +$ docker compose up --build -d +``` + +Open a browser and view the application at [http://localhost:3000](http://localhost:3000). + +In the terminal, run the following command to stop the application. + +```console +$ docker compose down +``` + +For more information about Compose commands, see the [Compose CLI +reference](/reference/cli/docker/compose/). + +### Summary + +In this section, you learned how to containerize and run a Node.js application using Docker. + +Related information: + +- [Docker Hardened Images](/dhi/) +- [Dockerfile reference](/reference/dockerfile.md) +- [Multi-stage builds](/manuals/build/building/multi-stage.md) +- [Docker Compose overview](/manuals/compose/_index.md) + +### Next steps + +In the next section, you'll take a look at how to set up a local development environment using Docker containers. + +## Use containers for Node.js development + +### Prerequisites + +Complete [Containerize a Node.js application](./). + +### Overview + +Once your application runs in a container, the next step is making the container part of your everyday development workflow. Code changes should show up quickly, and services your app depends on, like databases, should run right alongside it. + +In this section, you'll adapt the Dockerfile for local development by renaming the `builder` stage to `dev` and pointing Compose at it. You'll also update the application to connect to a PostgreSQL database, add a database service to `compose.yaml`, persist data in a named volume, enable Compose Watch so changes in your editor are picked up without a manual rebuild, and set up Node.js debugging so you can attach VS Code or Chrome DevTools to the running container. + +### Update the application + +You'll update your application to connect to a PostgreSQL database. Continue working in your `nodejs-docker-example` directory. + +Replace `src/index.ts` and `package.json` with the following contents. The file browser shows only the files that change in this step. + +> [!NOTE] +> +> The application won't run yet after this step. It tries to connect to a +> PostgreSQL database that doesn't exist. The next two sections add the +> database service and the Docker configuration needed to run everything +> together. + +{{< files name="nodejs-docker-example" >}} + +{{< file path="src/index.ts" status="modified" >}} +```typescript +// Express application backed by a PostgreSQL database. +// Creates a heroes table at startup. +// Endpoints: GET / (greeting), GET /health (health check), POST /heroes/ (create), GET /heroes/ (list). +// See https://expressjs.com/ and https://node-postgres.com/ + +import express, { type Request, type Response } from 'express'; +import { Pool } from 'pg'; +import { readFileSync } from 'fs'; + +const app = express(); +const port = parseInt(process.env.PORT ?? '3000', 10); + +app.use(express.json()); + +function getPassword(): string { + const passwordFile = process.env.POSTGRES_PASSWORD_FILE; + if (passwordFile) { + return readFileSync(passwordFile, 'utf8').trim(); + } + return process.env.POSTGRES_PASSWORD ?? ''; +} + +const pool = new Pool({ + host: process.env.POSTGRES_SERVER, + port: 5432, + database: process.env.POSTGRES_DB, + user: process.env.POSTGRES_USER, + password: getPassword(), +}); + +pool + .query( + `CREATE TABLE IF NOT EXISTS heroes ( + id SERIAL PRIMARY KEY, + name TEXT NOT NULL, + secret_name TEXT NOT NULL, + age INTEGER + )`, + ) + .catch(console.error); + +app.get('/', (_req: Request, res: Response) => { + res.json({ message: 'Hello World' }); +}); + +app.get('/health', (_req: Request, res: Response) => { + res.json({ status: 'ok' }); +}); + +app.post('/heroes/', async (req: Request, res: Response) => { + const { name, secret_name, age } = req.body as { + name: string; + secret_name: string; + age?: number; + }; + const result = await pool.query( + 'INSERT INTO heroes (name, secret_name, age) VALUES ($1, $2, $3) RETURNING *', + [name, secret_name, age], + ); + res.json(result.rows[0]); +}); + +app.get('/heroes/', async (_req: Request, res: Response) => { + const result = await pool.query('SELECT * FROM heroes'); + res.json(result.rows); +}); + +app.listen(port, () => { + console.log(`Server listening on port ${port}`); +}); +``` +{{< /file >}} + +{{< file path="package.json" status="modified" hl_lines="13,18" >}} +```json +{ + "name": "nodejs-docker-example", + "version": "1.0.0", + "description": "A minimal Node.js TypeScript application.", + "main": "dist/index.js", + "scripts": { + "build": "tsc", + "start": "node dist/index.js", + "dev": "tsx watch src/index.ts" + }, + "dependencies": { + "express": "^4.21.2", + "pg": "^8.16.0" + }, + "devDependencies": { + "@types/express": "^4.17.21", + "@types/node": "^22.0.0", + "@types/pg": "^8.11.0", + "tsx": "^4.19.3", + "typescript": "^5.8.3" + } +} +``` +{{< /file >}} + +{{< /files >}} + +### Update Docker assets + +Replace `Dockerfile` and `compose.yaml` with the following. + +{{< files name="nodejs-docker-example" >}} + +{{< file path="Dockerfile" status="modified" hl_lines="12,34,37,63" >}} +```dockerfile +# syntax=docker/dockerfile:1 + +# Comments are provided throughout this file to help you get started. +# If you need more help, visit the Dockerfile reference guide at +# https://docs.docker.com/go/dockerfile-reference/ + +# This Dockerfile uses Docker Hardened Images (DHI) for enhanced security. +# For more information, see https://docs.docker.com/dhi/ + +# Development stage: install all dependencies, compile TypeScript, and +# serve with hot-reload. Used directly in development via compose.yaml. +FROM dhi.io/node:24-alpine3.23-dev AS dev + +WORKDIR /app + +# Install dependencies as a separate step to take advantage of Docker's +# caching. Leverage a cache mount to /root/.npm to speed up subsequent +# builds. Leverage a bind mount to package.json to avoid having to copy +# it into this layer. +RUN --mount=type=cache,target=/root/.npm \ + --mount=type=bind,source=package.json,target=package.json \ + npm install +# Once you create a package-lock.json by running npm install locally, switch to npm ci and bind both files: +# RUN --mount=type=cache,target=/root/.npm \ +# --mount=type=bind,source=package.json,target=package.json \ +# --mount=type=bind,source=package-lock.json,target=package-lock.json \ +# npm ci + +# Copy the source code into the container and compile TypeScript. +COPY . . +RUN npm run build + +# Expose the port that the application listens on. +EXPOSE 3000 + +# Run the application in development mode. +CMD ["npm", "run", "dev"] + + +# Deps stage: install production dependencies only. +FROM dhi.io/node:24-alpine3.23-dev AS deps + +WORKDIR /app + +RUN --mount=type=cache,target=/root/.npm \ + --mount=type=bind,source=package.json,target=package.json \ + npm install --omit=dev +# Once you create a package-lock.json by running npm install locally, switch to npm ci and bind both files: +# RUN --mount=type=cache,target=/root/.npm \ +# --mount=type=bind,source=package.json,target=package.json \ +# --mount=type=bind,source=package-lock.json,target=package-lock.json \ +# npm ci --omit=dev + + +# Runner stage: minimal runtime image with compiled app and production deps. +FROM dhi.io/node:24-alpine3.23 AS runner + +ENV PATH=/app/node_modules/.bin:$PATH + +WORKDIR /app + +COPY --from=deps --chown=node:node /app/node_modules ./node_modules +COPY --from=dev --chown=node:node /app/dist ./dist + +# Expose the port that the application listens on. +EXPOSE 3000 + +# Run the application. +CMD ["node", "dist/index.js"] +``` +{{< /file >}} + +{{< file path="compose.yaml" status="modified" hl_lines="8" >}} +```yaml +services: + # Application service. The `target: dev` line builds the development + # image (includes tsx and dev tooling); the runner stage of the + # Dockerfile is unused in development. + server: + build: + context: . + target: dev + ports: + - 3000:3000 +``` +{{< /file >}} + +{{< /files >}} + +#### About these changes + +The `builder` stage from containerize is renamed to `dev` and gains `EXPOSE 3000` and `CMD ["npm", "run", "dev"]`, which runs `tsx watch` for hot-reload. The `deps` and `runner` stages are unchanged. + +In `compose.yaml`, the new `target: dev` line tells Compose to build and run the `dev` stage during development. Unlike the production image, the development image includes `tsx` and other dev tooling. If you need a shell in a running production container, use [Docker Debug](/reference/cli/docker/debug/) instead. + +The build step runs `tsc`, which compiles each TypeScript file into a corresponding JavaScript file. [esbuild](https://esbuild.github.io/) is a popular alternative that bundles everything into a single output file and builds significantly faster. To switch, replace the `tsc` call in `package.json` with an esbuild command and update the `COPY --from=dev` path in the `runner` stage to match esbuild's output. + +### Add a local database and persist data + +You can use containers to set up local services, like a database. In this section, you'll update the `compose.yaml` file to define a database service and a volume to persist data, and add a `db/password.txt` file that holds the database password. + +{{< files name="nodejs-docker-example" >}} + +{{< file path="compose.yaml" status="modified" hl_lines="11-46" >}} +```yaml +services: + # Application service. The `target: dev` line builds the development + # image (includes tsx and dev tooling); the runner stage of the + # Dockerfile is unused in development. + server: + build: + context: . + target: dev + ports: + - 3000:3000 + environment: + - POSTGRES_SERVER=db + - POSTGRES_USER=postgres + - POSTGRES_DB=example + - POSTGRES_PASSWORD_FILE=/run/secrets/db-password + depends_on: + db: + condition: service_healthy + secrets: + - db-password + # Database service. Reads the password from a Docker secret mounted at + # /run/secrets/db-password. Compose waits for the healthcheck to pass + # before starting the server, via the server's depends_on. + db: + image: dhi.io/postgres:18 + restart: always + user: postgres + secrets: + - db-password + volumes: + - db-data:/var/lib/postgresql + environment: + - POSTGRES_DB=example + - POSTGRES_PASSWORD_FILE=/run/secrets/db-password + expose: + - 5432 + healthcheck: + test: ["CMD", "pg_isready"] + interval: 10s + timeout: 5s + retries: 5 +volumes: + db-data: +secrets: + db-password: + file: db/password.txt +``` +{{< /file >}} + +{{< file path="db/password.txt" status="new" >}} +```text +mysecretpassword +``` +{{< /file >}} + +{{< /files >}} + +> [!NOTE] +> +> To learn more about the instructions in the Compose file, see [Compose file +> reference](/reference/compose-file/). + +Now, run the following `docker compose up` command to start your application. + +```console +$ docker compose up --build +``` + +Now test your API endpoint. Open a new terminal and make a request to the server using the curl commands. + +Create an object with a POST request: + +```console +$ curl -X 'POST' \ + 'http://localhost:3000/heroes/' \ + -H 'accept: application/json' \ + -H 'Content-Type: application/json' \ + -d '{ + "name": "my hero", + "secret_name": "austing", + "age": 12 +}' +``` + +You should receive the following response: + +```json +{ + "id": 1, + "name": "my hero", + "secret_name": "austing", + "age": 12 +} +``` + +Now make a GET request: + +```console +$ curl http://localhost:3000/heroes/ +``` + +You should receive the same response because it's the only object in the database. + +Press `ctrl`+`c` in the terminal to stop your application. + +### Automatically update services + +Use Compose Watch to automatically update your running Compose services as you edit and save your code. For more details about Compose Watch, see [Use Compose Watch](/manuals/compose/how-tos/file-watch.md). + +Open your `compose.yaml` file in an IDE or text editor and add the highlighted Compose Watch instructions. + +{{< files name="nodejs-docker-example" >}} + +{{< file path="compose.yaml" status="modified" hl_lines="21-27" >}} +```yaml +services: + # Application service. The `target: dev` line builds the development + # image (includes tsx and dev tooling); the runner stage of the + # Dockerfile is unused in development. + server: + build: + context: . + target: dev + ports: + - 3000:3000 + environment: + - POSTGRES_SERVER=db + - POSTGRES_USER=postgres + - POSTGRES_DB=example + - POSTGRES_PASSWORD_FILE=/run/secrets/db-password + depends_on: + db: + condition: service_healthy + secrets: + - db-password + develop: + watch: + - action: sync + path: ./src + target: /app/src + - action: rebuild + path: package.json + db: + image: dhi.io/postgres:18 + restart: always + user: postgres + secrets: + - db-password + volumes: + - db-data:/var/lib/postgresql + environment: + - POSTGRES_DB=example + - POSTGRES_PASSWORD_FILE=/run/secrets/db-password + expose: + - 5432 + healthcheck: + test: ["CMD", "pg_isready"] + interval: 10s + timeout: 5s + retries: 5 +volumes: + db-data: +secrets: + db-password: + file: db/password.txt +``` +{{< /file >}} + +{{< /files >}} + +Run the following command to run your application with Compose Watch. + +```console +$ docker compose watch +``` + +In a terminal, curl the application to get a response. + +```console +$ curl http://localhost:3000 +{"message":"Hello World"} +``` + +Any changes to the application's source files on your local machine will now be immediately reflected in the running container. + +Open `nodejs-docker-example/src/index.ts` in an IDE or text editor and update the `Hello World` string by adding a few more exclamation marks. + +```diff +- res.json({ message: 'Hello World' }); ++ res.json({ message: 'Hello World!!!' }); +``` + +Save the changes to `src/index.ts` and then wait a few seconds for the application to reload. Curl the application again and verify that the updated text appears. + +```console +$ curl http://localhost:3000 +{"message":"Hello World!!!"} +``` + +Press `ctrl`+`c` in the terminal to stop your application. + +### Debug your application + +`tsx watch` supports the Node.js inspector protocol, so you can attach a debugger from VS Code or Chrome DevTools and set breakpoints directly in your TypeScript source files. + +Update the `dev` script in `package.json` to start the inspector. The `--inspect=0.0.0.0:9229` flag tells Node.js to listen for debugger connections on all network interfaces at port 9229. Using `0.0.0.0` rather than `localhost` is necessary so the debugger is reachable from outside the container. Also expose the debug port in `compose.yaml`, and add a `.vscode/launch.json` file that tells VS Code how to attach to the running inspector. + +{{< files name="nodejs-docker-example" >}} + +{{< file path="package.json" status="modified" hl_lines="9" >}} +```json +{ + "name": "nodejs-docker-example", + "version": "1.0.0", + "description": "A minimal Node.js TypeScript application.", + "main": "dist/index.js", + "scripts": { + "build": "tsc", + "start": "node dist/index.js", + "dev": "tsx watch --inspect=0.0.0.0:9229 src/index.ts" + }, + "dependencies": { + "express": "^4.21.2", + "pg": "^8.16.0" + }, + "devDependencies": { + "@types/express": "^4.17.21", + "@types/node": "^22.0.0", + "@types/pg": "^8.11.0", + "tsx": "^4.19.3", + "typescript": "^5.8.3" + } +} +``` +{{< /file >}} + +{{< file path=".vscode/launch.json" status="new" >}} +```json +{ + "version": "0.2.0", + "configurations": [ + { + "name": "Attach to Docker Container", + "type": "node", + "request": "attach", + "port": 9229, + "address": "localhost", + "localRoot": "${workspaceFolder}", + "remoteRoot": "/app", + "protocol": "inspector", + "restart": true, + "sourceMaps": true, + "skipFiles": ["/**"] + } + ] +} +``` +{{< /file >}} + +{{< file path="compose.yaml" status="modified" hl_lines="11" >}} +```yaml +services: + # Application service. The `target: dev` line builds the development + # image (includes tsx and dev tooling); the runner stage of the + # Dockerfile is unused in development. + server: + build: + context: . + target: dev + ports: + - 3000:3000 + - 9229:9229 + environment: + - POSTGRES_SERVER=db + - POSTGRES_USER=postgres + - POSTGRES_DB=example + - POSTGRES_PASSWORD_FILE=/run/secrets/db-password + depends_on: + db: + condition: service_healthy + secrets: + - db-password + develop: + watch: + - action: sync + path: ./src + target: /app/src + - action: rebuild + path: package.json + db: + image: dhi.io/postgres:18 + restart: always + user: postgres + secrets: + - db-password + volumes: + - db-data:/var/lib/postgresql + environment: + - POSTGRES_DB=example + - POSTGRES_PASSWORD_FILE=/run/secrets/db-password + expose: + - 5432 + healthcheck: + test: ["CMD", "pg_isready"] + interval: 10s + timeout: 5s + retries: 5 +volumes: + db-data: +secrets: + db-password: + file: db/password.txt +``` +{{< /file >}} + +{{< /files >}} + +Rebuild and restart with the updated configuration: + +```console +$ docker compose up --build +``` + +When the inspector is ready, you'll see a line like the following in the logs: + +```text +Debugger listening on ws://0.0.0.0:9229/... +``` + +#### VS Code + +With `.vscode/launch.json` in place, attach the debugger using the Debug panel. + +Open the Debug panel (`Ctrl+Shift+D` on Windows and Linux, `Cmd+Shift+D` on Mac), select **Attach to Docker Container**, and press `F5`. You can now set breakpoints in your TypeScript source files under `src/`. + +#### Chrome DevTools + +You can also use the built-in Node.js inspector in Chrome without any editor setup. + +1. Open Chrome and go to `chrome://inspect`. + +2. Select **Configure** and add `localhost:9229`. + +3. When your Node.js target appears in the list, select **inspect**. + +#### Troubleshoot the debugger + +If the debugger doesn't connect, verify the container is running and the port is mapped correctly: + +```console +$ docker compose ps +$ docker compose logs server +``` + +The logs should include a line like: + +```text +Debugger listening on ws://0.0.0.0:9229/... +``` + +If that line is missing, confirm the `dev` script in `package.json` includes `--inspect=0.0.0.0:9229` and that `9229:9229` appears in the `ports` list for the `server` service in `compose.yaml`. + +For more details about Node.js debugging, see the [Node.js debugging guide](https://nodejs.org/en/docs/guides/debugging-getting-started). + +### Summary + +In this section, you set up a Compose file with a local database and persistent storage, set up Compose Watch to automatically sync code changes, and configured a debugger that attaches from VS Code and Chrome DevTools. + +Related information: + +- [Compose file reference](/reference/compose-file/) +- [Compose secrets](/reference/compose-file/secrets.md) +- [Compose Watch](/manuals/compose/how-tos/file-watch.md) +- [Multi-stage builds](/manuals/build/building/multi-stage.md) +- [Node.js debugging guide](https://nodejs.org/en/docs/guides/debugging-getting-started) + +### Next steps + +In the next section, you'll learn how to run tests using Docker. + +## Run Node.js tests in a container + +### Prerequisites + +Complete all the previous sections of this guide, starting with [Containerize a Node.js application](./). + +### Overview + +Testing is a core part of building reliable software. Docker makes it easy to +run your tests in the same environment used in CI and production, so failures +are caught before they reach your users. + +In this section, you'll add [Vitest](https://vitest.dev/) to the project and +run tests both locally and inside a container. + +### Update the application + +You'll refactor `src/index.ts` to export the Express `app` instance so tests +can import it without starting a server. Add a test file and update +`package.json` to add Vitest and a test runner for HTTP requests. The file browser shows only the files that change in this step. + +{{< files name="nodejs-docker-example" >}} + +{{< file path="src/index.ts" status="modified" hl_lines="10,31,70-75" >}} +```typescript +// Express application backed by a PostgreSQL database. +// Creates a heroes table at startup. +// Endpoints: GET / (greeting), GET /health (health check), POST /heroes/ (create), GET /heroes/ (list). +// See https://expressjs.com/ and https://node-postgres.com/ + +import express, { type Request, type Response } from 'express'; +import { Pool } from 'pg'; +import { readFileSync } from 'fs'; + +export const app = express(); +const port = parseInt(process.env.PORT ?? '3000', 10); + +app.use(express.json()); + +function getPassword(): string { + const passwordFile = process.env.POSTGRES_PASSWORD_FILE; + if (passwordFile) { + return readFileSync(passwordFile, 'utf8').trim(); + } + return process.env.POSTGRES_PASSWORD ?? ''; +} + +const pool = new Pool({ + host: process.env.POSTGRES_SERVER, + port: 5432, + database: process.env.POSTGRES_DB, + user: process.env.POSTGRES_USER, + password: getPassword(), +}); + +if (process.env.POSTGRES_SERVER) { + pool + .query( + `CREATE TABLE IF NOT EXISTS heroes ( + id SERIAL PRIMARY KEY, + name TEXT NOT NULL, + secret_name TEXT NOT NULL, + age INTEGER + )`, + ) + .catch(console.error); +} + +app.get('/', (_req: Request, res: Response) => { + res.json({ message: 'Hello World' }); +}); + +app.get('/health', (_req: Request, res: Response) => { + res.json({ status: 'ok' }); +}); + +app.post('/heroes/', async (req: Request, res: Response) => { + const { name, secret_name, age } = req.body as { + name: string; + secret_name: string; + age?: number; + }; + const result = await pool.query( + 'INSERT INTO heroes (name, secret_name, age) VALUES ($1, $2, $3) RETURNING *', + [name, secret_name, age], + ); + res.json(result.rows[0]); +}); + +app.get('/heroes/', async (_req: Request, res: Response) => { + const result = await pool.query('SELECT * FROM heroes'); + res.json(result.rows); +}); + +// Only start the server when this file is run directly. +if (require.main === module) { + app.listen(port, () => { + console.log(`Server listening on port ${port}`); + }); +} +``` +{{< /file >}} + +{{< file path="src/index.test.ts" status="new" >}} +```typescript +// Unit tests for the Express application. +// Tests the root endpoint without starting a server. +// See https://vitest.dev/ for the test framework reference. + +import { describe, it, expect } from 'vitest'; +import request from 'supertest'; +import { app } from './index'; + +describe('GET /', () => { + it('returns a JSON greeting', async () => { + const response = await request(app).get('/'); + expect(response.status).toBe(200); + expect(response.body).toEqual({ message: 'Hello World' }); + }); +}); +``` +{{< /file >}} + +{{< file path="package.json" status="modified" hl_lines="10,20-22" >}} +```json +{ + "name": "nodejs-docker-example", + "version": "1.0.0", + "description": "A minimal Node.js TypeScript application.", + "main": "dist/index.js", + "scripts": { + "build": "tsc", + "start": "node dist/index.js", + "dev": "tsx watch src/index.ts", + "test": "vitest run" + }, + "dependencies": { + "express": "^4.21.2", + "pg": "^8.16.0" + }, + "devDependencies": { + "@types/express": "^4.17.21", + "@types/node": "^22.0.0", + "@types/pg": "^8.11.0", + "supertest": "^7.0.0", + "@types/supertest": "^6.0.0", + "tsx": "^4.19.3", + "typescript": "^5.8.3", + "vitest": "^3.0.0" + } +} +``` +{{< /file >}} + +{{< /files >}} + +### Run tests locally + +Run the following command to run the tests locally: + +```console +$ npm install +$ npm test +``` + +You should see output like the following: + +```console + RUN v3.0.0 /app + + ✓ src/index.test.ts (1) + ✓ GET / (1) + ✓ returns a JSON greeting + + Test Files 1 passed (1) + Tests 1 passed (1) + Start at 12:00:00 + Duration 500ms +``` + +### Run tests in a container + +Run the tests using the dev stage of your Dockerfile: + +```console +$ docker compose run --build --rm --no-deps server npm test +``` + +The `--no-deps` flag skips starting the database, since the unit tests don't require it. The `--rm` flag removes the container when the tests finish. + +You should see the same test output as when running locally. + +### Run tests when building + +To run tests during the Docker build process, add a `test` stage to your Dockerfile that runs after the dev stage. + +```dockerfile {hl_lines="32-36"} +FROM dhi.io/node:24-alpine3.23-dev AS dev + +WORKDIR /app + +RUN --mount=type=cache,target=/root/.npm \ + --mount=type=bind,source=package.json,target=package.json \ + npm install + +COPY . . +RUN npm run build + +EXPOSE 3000 +CMD ["npm", "run", "dev"] + + +FROM dhi.io/node:24-alpine3.23-dev AS deps +WORKDIR /app +RUN --mount=type=cache,target=/root/.npm \ + --mount=type=bind,source=package.json,target=package.json \ + npm install --omit=dev + +FROM dhi.io/node:24-alpine3.23 AS runner +ENV PATH=/app/node_modules/.bin:$PATH +WORKDIR /app +COPY --from=deps --chown=node:node /app/node_modules ./node_modules +COPY --from=dev --chown=node:node /app/dist ./dist + +EXPOSE 3000 +CMD ["node", "dist/index.js"] + + +FROM dev AS test + +ENV CI=true + +CMD ["npm", "test"] +``` + +Then build and run the test stage: + +```console +$ docker build --target test -t nodejs-app-test . +$ docker run --rm nodejs-app-test +``` + +### Summary + +In this section, you learned how to run tests when developing locally and inside a container. + +Related information: + +- [Dockerfile reference](/reference/dockerfile/) +- [Compose file reference](/compose/compose-file/) +- [`docker compose run` CLI reference](/reference/cli/docker/compose/run/) + +### Next steps + +In the next section, you'll learn how to set up a CI/CD pipeline using GitHub Actions. + +## Automate your builds with GitHub Actions + +### Prerequisites + +Complete all the previous sections of this guide, starting with [Containerize a Node.js application](./). You must have a [GitHub](https://github.com/signup) account and a verified [Docker](https://hub.docker.com/signup) account to complete this section. + +If you haven't created a [GitHub repository](https://github.com/new) for your project yet, do that now. After creating the repository, [add a remote](https://docs.github.com/en/get-started/getting-started-with-git/managing-remote-repositories) and make sure you can commit and [push your code](https://docs.github.com/en/get-started/using-git/pushing-commits-to-a-remote-repository#about-git-push) to GitHub. + +1. In your project's GitHub repository, open **Settings**, and go to **Secrets and variables** > **Actions**. + +2. Under the **Variables** tab, create a new **Repository variable** named `DOCKER_USERNAME` with your Docker ID as the value. + +3. Create a new [Personal Access Token (PAT)](/manuals/security/access-tokens.md#create-an-access-token) for Docker Hub. You can name this token `docker-tutorial`. Make sure access permissions include Read and Write. + +4. Add the PAT as a **Repository secret** in your GitHub repository, with the name `DOCKERHUB_TOKEN`. + +### Overview + +GitHub Actions is a CI/CD automation tool built into GitHub. A workflow is a YAML file that tells GitHub which jobs to run when something happens in your repository, like a push to a branch or a pull request opening. Workflows live in the `.github/workflows/` directory of your repository. + +In this section, you'll add a workflow that runs your tests on every push to the main branch, then builds your Docker image and pushes it to Docker Hub. + +### Define the GitHub Actions workflow + +You can create a GitHub Actions workflow by creating a YAML file in the `.github/workflows/` directory of your repository. Use your favorite text editor or the GitHub web interface. + +If you prefer to use the GitHub web interface: + +1. Go to your repository on GitHub and select the **Actions** tab. + +2. Select **set up a workflow yourself**. + + This takes you to a page for creating a new GitHub Actions workflow file in your repository. By default, the file is created under `.github/workflows/main.yml`. Change the filename to `build.yml`. + +If you prefer to use your text editor, create a new file named `build.yml` in the `.github/workflows/` directory of your repository. + +Add the following content to the file: + +{{< files name="nodejs-docker-example" >}} + +{{< file path=".github/workflows/build.yml" status="new" >}} +```yaml +# GitHub Actions workflow that runs on every push to main. +# - test: runs Vitest unit tests inside a container. +# - build_and_push: signs in to Docker Hub and the DHI registry, then +# builds and pushes the image. +name: Build and push Docker image + +on: + push: + branches: + - main + +jobs: + test: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@{{% param "checkout_action_version" %}} + + - name: Run tests + run: docker build --target test -t nodejs-test . && docker run --rm nodejs-test + + build_and_push: + runs-on: ubuntu-latest + needs: test + steps: + - uses: actions/checkout@{{% param "checkout_action_version" %}} + + - name: Login to Docker Hub + uses: docker/login-action@{{% param "login_action_version" %}} + with: + username: ${{ vars.DOCKER_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + + - name: Login to Docker Hardened Images + uses: docker/login-action@{{% param "login_action_version" %}} + with: + registry: dhi.io + username: ${{ vars.DOCKER_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@{{% param "setup_buildx_action_version" %}} + + - name: Build and push + uses: docker/build-push-action@{{% param "build_push_action_version" %}} + with: + push: true + tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest +``` +{{< /file >}} + +{{< /files >}} + +The workflow has two jobs: + +1. **test**: Builds the `test` stage of the Dockerfile and runs it. If tests fail, the workflow stops and `build_and_push` doesn't run. +2. **build_and_push**: Signs in to Docker Hub and the DHI registry, then builds and pushes the image. + +### Run the workflow + +Commit the changes and push them to the `main` branch. This workflow runs every time you push changes to `main`. You can find more information about workflow triggers in the [GitHub documentation](https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows). + +Go to the **Actions** tab of your GitHub repository. It displays the workflow. Selecting the workflow shows you the breakdown of all the steps. + +When the workflow is complete, go to your [repositories on Docker Hub](https://hub.docker.com/repositories). If you see the new repository in that list, the GitHub Actions workflow successfully pushed the image to Docker Hub. + +### Summary + +In this section, you learned how to set up a GitHub Actions workflow for your Node.js application that includes: + +- Running Vitest unit tests inside a container +- Building and pushing Docker images + +Related information: + +- [Introduction to GitHub Actions](/guides/gha.md) +- [Docker Build GitHub Actions](/manuals/build/ci/github-actions/_index.md) +- [docker/login-action](https://github.com/docker/login-action) +- [docker/build-push-action](https://github.com/docker/build-push-action) +- [Create a Docker Hub access token](/manuals/security/access-tokens.md#create-an-access-token) + +### Next steps + +In the next section, you'll learn how to inspect and generate supply chain +attestations for your image. See [Secure your supply chain](./). + +## Secure your Node.js image supply chain + +### Prerequisites + +Complete [Automate your builds with GitHub Actions](./). + +### Overview + +When you ship a container image, what's inside it and where it came from +matters. Supply chain attestations are signed records that answer questions +like which packages are in the image, what vulnerabilities affect them, how +the image was built, and what security checks it passed. + +In this section, you'll inspect the attestations that ship with your Docker +Hardened Image base, generate your own SBOM and provenance attestations +during CI, and pin the base image by digest so your builds are reproducible. + +The inspection commands in this topic are shown manually so you can see what +each one returns. In a real workflow you'd automate these checks with +[Docker Scout](/scout/), which runs the same scans on every push, +enforces policies in CI, and surfaces results in your registry and pull +requests. + +### Inspect the base image attestations + +Docker Hardened Images are built to SLSA Build Level 3 and ship with a set of +signed attestations covering bill-of-materials, vulnerabilities, build +provenance, and security scans. See +[DHI attestations](/manuals/dhi/core-concepts/attestations.md) for the full +list of types and how to verify their signatures with Cosign. + +List all the attestations available on the Node.js DHI: + +```console +$ docker scout attest list registry://dhi.io/node:24-alpine3.23-dev +``` + +View the SBOM: + +```console +$ docker scout sbom registry://dhi.io/node:24-alpine3.23-dev +``` + +Check known vulnerabilities: + +```console +$ docker scout cves registry://dhi.io/node:24-alpine3.23-dev +``` + +> [!NOTE] +> +> The `registry://` prefix forces `docker scout` to fetch the image and its +> attestations from the registry instead of reading a locally pulled copy. If +> you've already pulled or built against the base image, the local copy +> doesn't have the attached attestations, so the prefix is required to see +> them. + +When you base your own image on a DHI image, these attestations stay attached to the base layer in the registry. Tools that inspect your image can follow the chain back to the DHI source. + +### Generate attestations for your image + +Update your GitHub Actions workflow to attach SBOM and provenance attestations to the image you push. + +Edit `.github/workflows/build.yml` and update the build-and-push step: + +```yaml {hl_lines="6-7"} +- name: Build and push Docker image + uses: docker/build-push-action@{{% param "build_push_action_version" %}} + with: + context: . + push: true + sbom: true + provenance: mode=max + tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest +``` + +- `sbom: true` tells BuildKit to scan the built image and attach an SBOM attestation. +- `provenance: mode=max` records detailed build provenance, including the source repository, commit, and build parameters. + +The next time your workflow runs, the pushed image will carry these attestations alongside the image manifest in the registry. + +### Inspect your pushed image's attestations + +After your workflow pushes the image, inspect it the same way you inspected the base image: + +```console +$ docker scout attest list registry://DOCKER_USERNAME/REPO_NAME:latest +$ docker scout sbom registry://DOCKER_USERNAME/REPO_NAME:latest +``` + +The SBOM includes packages from every layer, including those inherited from `dhi.io/node:24-alpine3.23-dev`. The provenance record references the DHI base image by digest, so consumers of your image can trace the build chain back to the DHI source. + +### Pin the base image by digest + +Image tags like `dhi.io/node:24-alpine3.23-dev` move over time as new patches land. For reproducible builds, pin to an immutable digest. + +Look up the digest for each image: + +```console +$ docker buildx imagetools inspect dhi.io/node:24-alpine3.23-dev --format "{{ .Manifest.Digest }}" +sha256:2bf01111c7dfe429362f64b3977f0cd6e63ff39023012f88487dec7e83aa26ca +$ docker buildx imagetools inspect dhi.io/node:24-alpine3.23 --format "{{ .Manifest.Digest }}" +sha256:868827fd45c6a01f7f3337ba7ff3f48ebb14da10d8cf3d347f98ded5481317a5 +``` + +Each digest is a 64-character hex string. Update your `Dockerfile` to reference each digest on its corresponding `FROM` line: + +```dockerfile +FROM dhi.io/node:24-alpine3.23-dev@sha256:2bf01111c7dfe429362f64b3977f0cd6e63ff39023012f88487dec7e83aa26ca AS dev +# ... +FROM dhi.io/node:24-alpine3.23@sha256:868827fd45c6a01f7f3337ba7ff3f48ebb14da10d8cf3d347f98ded5481317a5 AS runner +``` + +> [!TIP] +> +> Pinning by digest also pins you to that image's vulnerabilities. Use [Dependabot](https://docs.github.com/en/code-security/dependabot) or [Renovate](https://docs.renovatebot.com/) to automate digest updates so you get a PR when a new patched image is available, with a changelog to review before merging. + +### Summary + +In this section, you learned how to: + +- Inspect the supply chain attestations that ship with the DHI base image, including SBOMs, CVE reports, and build provenance +- Generate SBOM and provenance attestations for your own image in CI +- Pin base images by digest for reproducible builds + +Related information: + +- [DHI attestations](/manuals/dhi/core-concepts/attestations.md) +- [Verify a Docker Hardened Image](/manuals/dhi/how-to/verify.md) +- [Docker Scout](/scout/) +- [Build attestations](/manuals/build/metadata/attestations/_index.md) + +### Next steps + +In the next section, you'll deploy your application to Kubernetes. + +## Deploy your Node.js application + +### Prerequisites + +- Complete all the previous sections of this guide, starting with [Containerize a Node.js application](./). +- [Turn on Kubernetes](/manuals/desktop/use-desktop/kubernetes.md#enable-kubernetes) in Docker Desktop. + +### Overview + +In this section, you'll deploy your containerized Node.js application to a local Kubernetes cluster using Docker Desktop. You'll create a Kubernetes manifest that describes how the application should run, including the application deployment, the PostgreSQL database, and the services that connect them. + +### Create a Kubernetes manifest + +Create a new file called `nodejs-docker-example-kubernetes.yaml` in your project root: + +{{< files name="nodejs-docker-example" >}} + +{{< file path="nodejs-docker-example-kubernetes.yaml" status="new" >}} +```yaml +apiVersion: v1 +kind: Namespace +metadata: + name: nodejs-docker-example + +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: app-config + namespace: nodejs-docker-example +data: + POSTGRES_SERVER: 'postgres' + POSTGRES_DB: 'example' + POSTGRES_USER: 'postgres' + +--- +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: postgres-pvc + namespace: nodejs-docker-example +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: postgres + namespace: nodejs-docker-example +spec: + replicas: 1 + selector: + matchLabels: + app: postgres + template: + metadata: + labels: + app: postgres + spec: + containers: + - name: postgres + image: dhi.io/postgres:18 + ports: + - containerPort: 5432 + env: + - name: POSTGRES_DB + valueFrom: + configMapKeyRef: + name: app-config + key: POSTGRES_DB + - name: POSTGRES_USER + valueFrom: + configMapKeyRef: + name: app-config + key: POSTGRES_USER + - name: POSTGRES_PASSWORD + valueFrom: + secretKeyRef: + name: app-secrets + key: db-password + volumeMounts: + - name: postgres-storage + mountPath: /var/lib/postgresql + readinessProbe: + exec: + command: [pg_isready] + initialDelaySeconds: 5 + periodSeconds: 5 + volumes: + - name: postgres-storage + persistentVolumeClaim: + claimName: postgres-pvc + +--- +apiVersion: v1 +kind: Service +metadata: + name: postgres + namespace: nodejs-docker-example +spec: + type: ClusterIP + ports: + - port: 5432 + targetPort: 5432 + selector: + app: postgres + +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: server + namespace: nodejs-docker-example +spec: + replicas: 2 + selector: + matchLabels: + app: server + template: + metadata: + labels: + app: server + spec: + containers: + - name: server + image: DOCKER_USERNAME/nodejs-docker-example:latest + ports: + - containerPort: 3000 + env: + - name: POSTGRES_SERVER + valueFrom: + configMapKeyRef: + name: app-config + key: POSTGRES_SERVER + - name: POSTGRES_DB + valueFrom: + configMapKeyRef: + name: app-config + key: POSTGRES_DB + - name: POSTGRES_USER + valueFrom: + configMapKeyRef: + name: app-config + key: POSTGRES_USER + - name: POSTGRES_PASSWORD + valueFrom: + secretKeyRef: + name: app-secrets + key: db-password + readinessProbe: + httpGet: + path: /health + port: 3000 + initialDelaySeconds: 10 + periodSeconds: 5 + +--- +apiVersion: v1 +kind: Service +metadata: + name: server + namespace: nodejs-docker-example +spec: + type: ClusterIP + ports: + - port: 3000 + targetPort: 3000 + selector: + app: server +``` +{{< /file >}} + +{{< /files >}} + +Before applying the manifest, replace `DOCKER_USERNAME` in the `server` deployment's image field with your Docker Hub username. + +### Deploy to Kubernetes + +Apply the manifest to your local Kubernetes cluster: + +```console +$ kubectl apply -f nodejs-docker-example-kubernetes.yaml +``` + +You should see output confirming that the resources were created: + +```console +namespace/nodejs-docker-example created +configmap/app-config created +persistentvolumeclaim/postgres-pvc created +deployment.apps/postgres created +service/postgres created +deployment.apps/server created +service/server created +``` + +Then create the database secret from your password file: + +```console +$ kubectl create secret generic app-secrets \ + --namespace nodejs-docker-example \ + --from-file=db-password=db/password.txt +``` + +### Verify the deployment + +Check that your pods are running: + +```console +$ kubectl get pods -n nodejs-docker-example +``` + +Wait until all pods show `Running` in the STATUS column. Then verify your services: + +```console +$ kubectl get services -n nodejs-docker-example +``` + +### Access the application + +Use port forwarding to access the application from your local machine: + +```console +$ kubectl port-forward -n nodejs-docker-example service/server 3000:3000 +``` + +Open a new terminal and make a request to the application: + +```console +$ curl http://localhost:3000 +{"message":"Hello World"} +``` + +You can also create a hero: + +```console +$ curl -X POST http://localhost:3000/heroes/ \ + -H 'Content-Type: application/json' \ + -d '{"name": "my hero", "secret_name": "austing", "age": 12}' +``` + +### Clean up + +When you're done testing, remove the deployment: + +```console +$ kubectl delete -f nodejs-docker-example-kubernetes.yaml +``` + +### Summary + +In this section, you deployed your containerized Node.js application to Kubernetes. You created a manifest that defines the application and database deployments, applied it to a local cluster, and verified the application is accessible. + +Related information: + +- [Kubernetes documentation](https://kubernetes.io/docs/home/) +- [Deploy on Kubernetes with Docker Desktop](/manuals/desktop/use-desktop/kubernetes.md) +- [`kubectl` CLI reference](https://kubernetes.io/docs/reference/kubectl/) +- [Kubernetes Deployment resource](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) +- [Kubernetes Service resource](https://kubernetes.io/docs/concepts/services-networking/service/) diff --git a/content/guides/nodejs/run-tests.md b/content/guides/nodejs/run-tests.md deleted file mode 100644 index f67879b426a0..000000000000 --- a/content/guides/nodejs/run-tests.md +++ /dev/null @@ -1,264 +0,0 @@ ---- -title: Run Node.js tests in a container -linkTitle: Run your tests -weight: 30 -keywords: node.js, node, test, vitest -description: Learn how to run your Node.js tests in a container. -aliases: - - /language/nodejs/run-tests/ - - /guides/language/nodejs/run-tests/ ---- - -## Prerequisites - -Complete all the previous sections of this guide, starting with [Containerize a Node.js application](containerize.md). - -## Overview - -Testing is a core part of building reliable software. Docker makes it easy to -run your tests in the same environment used in CI and production, so failures -are caught before they reach your users. - -In this section, you'll add [Vitest](https://vitest.dev/) to the project and -run tests both locally and inside a container. - -## Update the application - -You'll refactor `src/index.ts` to export the Express `app` instance so tests -can import it without starting a server. Add a test file and update -`package.json` to add Vitest and a test runner for HTTP requests. The file browser shows only the files that change in this step. - -{{< files name="nodejs-docker-example" >}} - -{{< file path="src/index.ts" status="modified" hl_lines="10,31,70-75" >}} -```typescript -// Express application backed by a PostgreSQL database. -// Creates a heroes table at startup. -// Endpoints: GET / (greeting), GET /health (health check), POST /heroes/ (create), GET /heroes/ (list). -// See https://expressjs.com/ and https://node-postgres.com/ - -import express, { type Request, type Response } from 'express'; -import { Pool } from 'pg'; -import { readFileSync } from 'fs'; - -export const app = express(); -const port = parseInt(process.env.PORT ?? '3000', 10); - -app.use(express.json()); - -function getPassword(): string { - const passwordFile = process.env.POSTGRES_PASSWORD_FILE; - if (passwordFile) { - return readFileSync(passwordFile, 'utf8').trim(); - } - return process.env.POSTGRES_PASSWORD ?? ''; -} - -const pool = new Pool({ - host: process.env.POSTGRES_SERVER, - port: 5432, - database: process.env.POSTGRES_DB, - user: process.env.POSTGRES_USER, - password: getPassword(), -}); - -if (process.env.POSTGRES_SERVER) { - pool - .query( - `CREATE TABLE IF NOT EXISTS heroes ( - id SERIAL PRIMARY KEY, - name TEXT NOT NULL, - secret_name TEXT NOT NULL, - age INTEGER - )`, - ) - .catch(console.error); -} - -app.get('/', (_req: Request, res: Response) => { - res.json({ message: 'Hello World' }); -}); - -app.get('/health', (_req: Request, res: Response) => { - res.json({ status: 'ok' }); -}); - -app.post('/heroes/', async (req: Request, res: Response) => { - const { name, secret_name, age } = req.body as { - name: string; - secret_name: string; - age?: number; - }; - const result = await pool.query( - 'INSERT INTO heroes (name, secret_name, age) VALUES ($1, $2, $3) RETURNING *', - [name, secret_name, age], - ); - res.json(result.rows[0]); -}); - -app.get('/heroes/', async (_req: Request, res: Response) => { - const result = await pool.query('SELECT * FROM heroes'); - res.json(result.rows); -}); - -// Only start the server when this file is run directly. -if (require.main === module) { - app.listen(port, () => { - console.log(`Server listening on port ${port}`); - }); -} -``` -{{< /file >}} - -{{< file path="src/index.test.ts" status="new" >}} -```typescript -// Unit tests for the Express application. -// Tests the root endpoint without starting a server. -// See https://vitest.dev/ for the test framework reference. - -import { describe, it, expect } from 'vitest'; -import request from 'supertest'; -import { app } from './index'; - -describe('GET /', () => { - it('returns a JSON greeting', async () => { - const response = await request(app).get('/'); - expect(response.status).toBe(200); - expect(response.body).toEqual({ message: 'Hello World' }); - }); -}); -``` -{{< /file >}} - -{{< file path="package.json" status="modified" hl_lines="10,20-22" >}} -```json -{ - "name": "nodejs-docker-example", - "version": "1.0.0", - "description": "A minimal Node.js TypeScript application.", - "main": "dist/index.js", - "scripts": { - "build": "tsc", - "start": "node dist/index.js", - "dev": "tsx watch src/index.ts", - "test": "vitest run" - }, - "dependencies": { - "express": "^4.21.2", - "pg": "^8.16.0" - }, - "devDependencies": { - "@types/express": "^4.17.21", - "@types/node": "^22.0.0", - "@types/pg": "^8.11.0", - "supertest": "^7.0.0", - "@types/supertest": "^6.0.0", - "tsx": "^4.19.3", - "typescript": "^5.8.3", - "vitest": "^3.0.0" - } -} -``` -{{< /file >}} - -{{< /files >}} - -## Run tests locally - -Run the following command to run the tests locally: - -```console -$ npm install -$ npm test -``` - -You should see output like the following: - -```console - RUN v3.0.0 /app - - ✓ src/index.test.ts (1) - ✓ GET / (1) - ✓ returns a JSON greeting - - Test Files 1 passed (1) - Tests 1 passed (1) - Start at 12:00:00 - Duration 500ms -``` - -## Run tests in a container - -Run the tests using the dev stage of your Dockerfile: - -```console -$ docker compose run --build --rm --no-deps server npm test -``` - -The `--no-deps` flag skips starting the database, since the unit tests don't require it. The `--rm` flag removes the container when the tests finish. - -You should see the same test output as when running locally. - -## Run tests when building - -To run tests during the Docker build process, add a `test` stage to your Dockerfile that runs after the dev stage. - -```dockerfile {hl_lines="32-36"} -FROM dhi.io/node:24-alpine3.23-dev AS dev - -WORKDIR /app - -RUN --mount=type=cache,target=/root/.npm \ - --mount=type=bind,source=package.json,target=package.json \ - npm install - -COPY . . -RUN npm run build - -EXPOSE 3000 -CMD ["npm", "run", "dev"] - - -FROM dhi.io/node:24-alpine3.23-dev AS deps -WORKDIR /app -RUN --mount=type=cache,target=/root/.npm \ - --mount=type=bind,source=package.json,target=package.json \ - npm install --omit=dev - -FROM dhi.io/node:24-alpine3.23 AS runner -ENV PATH=/app/node_modules/.bin:$PATH -WORKDIR /app -COPY --from=deps --chown=node:node /app/node_modules ./node_modules -COPY --from=dev --chown=node:node /app/dist ./dist - -EXPOSE 3000 -CMD ["node", "dist/index.js"] - - -FROM dev AS test - -ENV CI=true - -CMD ["npm", "test"] -``` - -Then build and run the test stage: - -```console -$ docker build --target test -t nodejs-app-test . -$ docker run --rm nodejs-app-test -``` - -## Summary - -In this section, you learned how to run tests when developing locally and inside a container. - -Related information: - -- [Dockerfile reference](/reference/dockerfile/) -- [Compose file reference](/compose/compose-file/) -- [`docker compose run` CLI reference](/reference/cli/docker/compose/run/) - -## Next steps - -In the next section, you'll learn how to set up a CI/CD pipeline using GitHub Actions. diff --git a/content/guides/nodejs/secure-supply-chain.md b/content/guides/nodejs/secure-supply-chain.md deleted file mode 100644 index 504933bebb98..000000000000 --- a/content/guides/nodejs/secure-supply-chain.md +++ /dev/null @@ -1,141 +0,0 @@ ---- -title: Secure your Node.js image supply chain -linkTitle: Secure your supply chain -weight: 45 -keywords: node.js, node, sbom, provenance, attestations, docker scout, supply chain, security -description: Learn how to inspect, generate, and verify supply chain attestations for your Node.js container image. ---- - -## Prerequisites - -Complete [Automate your builds with GitHub Actions](configure-github-actions.md). - -## Overview - -When you ship a container image, what's inside it and where it came from -matters. Supply chain attestations are signed records that answer questions -like which packages are in the image, what vulnerabilities affect them, how -the image was built, and what security checks it passed. - -In this section, you'll inspect the attestations that ship with your Docker -Hardened Image base, generate your own SBOM and provenance attestations -during CI, and pin the base image by digest so your builds are reproducible. - -The inspection commands in this topic are shown manually so you can see what -each one returns. In a real workflow you'd automate these checks with -[Docker Scout](/scout/), which runs the same scans on every push, -enforces policies in CI, and surfaces results in your registry and pull -requests. - -## Inspect the base image attestations - -Docker Hardened Images are built to SLSA Build Level 3 and ship with a set of -signed attestations covering bill-of-materials, vulnerabilities, build -provenance, and security scans. See -[DHI attestations](/manuals/dhi/core-concepts/attestations.md) for the full -list of types and how to verify their signatures with Cosign. - -List all the attestations available on the Node.js DHI: - -```console -$ docker scout attest list registry://dhi.io/node:24-alpine3.23-dev -``` - -View the SBOM: - -```console -$ docker scout sbom registry://dhi.io/node:24-alpine3.23-dev -``` - -Check known vulnerabilities: - -```console -$ docker scout cves registry://dhi.io/node:24-alpine3.23-dev -``` - -> [!NOTE] -> -> The `registry://` prefix forces `docker scout` to fetch the image and its -> attestations from the registry instead of reading a locally pulled copy. If -> you've already pulled or built against the base image, the local copy -> doesn't have the attached attestations, so the prefix is required to see -> them. - -When you base your own image on a DHI image, these attestations stay attached to the base layer in the registry. Tools that inspect your image can follow the chain back to the DHI source. - -## Generate attestations for your image - -Update your GitHub Actions workflow to attach SBOM and provenance attestations to the image you push. - -Edit `.github/workflows/build.yml` and update the build-and-push step: - -```yaml {hl_lines="6-7"} -- name: Build and push Docker image - uses: docker/build-push-action@{{% param "build_push_action_version" %}} - with: - context: . - push: true - sbom: true - provenance: mode=max - tags: ${{ vars.DOCKER_USERNAME }}/${{ github.event.repository.name }}:latest -``` - -- `sbom: true` tells BuildKit to scan the built image and attach an SBOM attestation. -- `provenance: mode=max` records detailed build provenance, including the source repository, commit, and build parameters. - -The next time your workflow runs, the pushed image will carry these attestations alongside the image manifest in the registry. - -## Inspect your pushed image's attestations - -After your workflow pushes the image, inspect it the same way you inspected the base image: - -```console -$ docker scout attest list registry://DOCKER_USERNAME/REPO_NAME:latest -$ docker scout sbom registry://DOCKER_USERNAME/REPO_NAME:latest -``` - -The SBOM includes packages from every layer, including those inherited from `dhi.io/node:24-alpine3.23-dev`. The provenance record references the DHI base image by digest, so consumers of your image can trace the build chain back to the DHI source. - -## Pin the base image by digest - -Image tags like `dhi.io/node:24-alpine3.23-dev` move over time as new patches land. For reproducible builds, pin to an immutable digest. - -Look up the digest for each image: - -```console -$ docker buildx imagetools inspect dhi.io/node:24-alpine3.23-dev --format "{{ .Manifest.Digest }}" -sha256:2bf01111c7dfe429362f64b3977f0cd6e63ff39023012f88487dec7e83aa26ca -$ docker buildx imagetools inspect dhi.io/node:24-alpine3.23 --format "{{ .Manifest.Digest }}" -sha256:868827fd45c6a01f7f3337ba7ff3f48ebb14da10d8cf3d347f98ded5481317a5 -``` - -Each digest is a 64-character hex string. Update your `Dockerfile` to reference each digest on its corresponding `FROM` line: - -```dockerfile -FROM dhi.io/node:24-alpine3.23-dev@sha256:2bf01111c7dfe429362f64b3977f0cd6e63ff39023012f88487dec7e83aa26ca AS dev -# ... -FROM dhi.io/node:24-alpine3.23@sha256:868827fd45c6a01f7f3337ba7ff3f48ebb14da10d8cf3d347f98ded5481317a5 AS runner -``` - -> [!TIP] -> -> Pinning by digest also pins you to that image's vulnerabilities. Use [Dependabot](https://docs.github.com/en/code-security/dependabot) or [Renovate](https://docs.renovatebot.com/) to automate digest updates so you get a PR when a new patched image is available, with a changelog to review before merging. - -## Summary - -In this section, you learned how to: - -- Inspect the supply chain attestations that ship with the DHI base image, including SBOMs, CVE reports, and build provenance -- Generate SBOM and provenance attestations for your own image in CI -- Pin base images by digest for reproducible builds - -Related information: - -- [DHI attestations](/manuals/dhi/core-concepts/attestations.md) -- [Verify a Docker Hardened Image](/manuals/dhi/how-to/verify.md) -- [Docker Scout](/scout/) -- [Build attestations](/manuals/build/metadata/attestations/_index.md) - -## Next steps - -In the next section, you'll deploy your application to Kubernetes. diff --git a/content/guides/php/_index.md b/content/guides/php/index.md similarity index 100% rename from content/guides/php/_index.md rename to content/guides/php/index.md diff --git a/content/guides/postgresql/_index.md b/content/guides/postgresql/index.md similarity index 100% rename from content/guides/postgresql/_index.md rename to content/guides/postgresql/index.md diff --git a/content/guides/python/_index.md b/content/guides/python/index.md similarity index 100% rename from content/guides/python/_index.md rename to content/guides/python/index.md diff --git a/content/guides/r/_index.md b/content/guides/r/index.md similarity index 100% rename from content/guides/r/_index.md rename to content/guides/r/index.md diff --git a/content/guides/rag-ollama/_index.md b/content/guides/rag-ollama/index.md similarity index 100% rename from content/guides/rag-ollama/_index.md rename to content/guides/rag-ollama/index.md diff --git a/content/guides/reactjs/_index.md b/content/guides/reactjs/index.md similarity index 100% rename from content/guides/reactjs/_index.md rename to content/guides/reactjs/index.md diff --git a/content/guides/ros2/_index.md b/content/guides/ros2/index.md similarity index 100% rename from content/guides/ros2/_index.md rename to content/guides/ros2/index.md diff --git a/content/guides/ruby/_index.md b/content/guides/ruby/index.md similarity index 100% rename from content/guides/ruby/_index.md rename to content/guides/ruby/index.md diff --git a/content/guides/rust/_index.md b/content/guides/rust/index.md similarity index 100% rename from content/guides/rust/_index.md rename to content/guides/rust/index.md diff --git a/content/guides/testcontainers-cloud/_index.md b/content/guides/testcontainers-cloud/index.md similarity index 100% rename from content/guides/testcontainers-cloud/_index.md rename to content/guides/testcontainers-cloud/index.md diff --git a/content/guides/testcontainers-dotnet-aspnet-core/_index.md b/content/guides/testcontainers-dotnet-aspnet-core/index.md similarity index 100% rename from content/guides/testcontainers-dotnet-aspnet-core/_index.md rename to content/guides/testcontainers-dotnet-aspnet-core/index.md diff --git a/content/guides/testcontainers-dotnet-getting-started/_index.md b/content/guides/testcontainers-dotnet-getting-started/index.md similarity index 100% rename from content/guides/testcontainers-dotnet-getting-started/_index.md rename to content/guides/testcontainers-dotnet-getting-started/index.md diff --git a/content/guides/testcontainers-go-getting-started/_index.md b/content/guides/testcontainers-go-getting-started/index.md similarity index 100% rename from content/guides/testcontainers-go-getting-started/_index.md rename to content/guides/testcontainers-go-getting-started/index.md diff --git a/content/guides/testcontainers-java-aws-localstack/_index.md b/content/guides/testcontainers-java-aws-localstack/index.md similarity index 100% rename from content/guides/testcontainers-java-aws-localstack/_index.md rename to content/guides/testcontainers-java-aws-localstack/index.md diff --git a/content/guides/testcontainers-java-getting-started/_index.md b/content/guides/testcontainers-java-getting-started/index.md similarity index 100% rename from content/guides/testcontainers-java-getting-started/_index.md rename to content/guides/testcontainers-java-getting-started/index.md diff --git a/content/guides/testcontainers-java-jooq-flyway/_index.md b/content/guides/testcontainers-java-jooq-flyway/index.md similarity index 100% rename from content/guides/testcontainers-java-jooq-flyway/_index.md rename to content/guides/testcontainers-java-jooq-flyway/index.md diff --git a/content/guides/testcontainers-java-keycloak-spring-boot/_index.md b/content/guides/testcontainers-java-keycloak-spring-boot/index.md similarity index 100% rename from content/guides/testcontainers-java-keycloak-spring-boot/_index.md rename to content/guides/testcontainers-java-keycloak-spring-boot/index.md diff --git a/content/guides/testcontainers-java-lifecycle/_index.md b/content/guides/testcontainers-java-lifecycle/index.md similarity index 100% rename from content/guides/testcontainers-java-lifecycle/_index.md rename to content/guides/testcontainers-java-lifecycle/index.md diff --git a/content/guides/testcontainers-java-micronaut-kafka/_index.md b/content/guides/testcontainers-java-micronaut-kafka/index.md similarity index 100% rename from content/guides/testcontainers-java-micronaut-kafka/_index.md rename to content/guides/testcontainers-java-micronaut-kafka/index.md diff --git a/content/guides/testcontainers-java-micronaut-wiremock/_index.md b/content/guides/testcontainers-java-micronaut-wiremock/index.md similarity index 100% rename from content/guides/testcontainers-java-micronaut-wiremock/_index.md rename to content/guides/testcontainers-java-micronaut-wiremock/index.md diff --git a/content/guides/testcontainers-java-mockserver/_index.md b/content/guides/testcontainers-java-mockserver/index.md similarity index 100% rename from content/guides/testcontainers-java-mockserver/_index.md rename to content/guides/testcontainers-java-mockserver/index.md diff --git a/content/guides/testcontainers-java-quarkus/_index.md b/content/guides/testcontainers-java-quarkus/index.md similarity index 100% rename from content/guides/testcontainers-java-quarkus/_index.md rename to content/guides/testcontainers-java-quarkus/index.md diff --git a/content/guides/testcontainers-java-replace-h2/_index.md b/content/guides/testcontainers-java-replace-h2/index.md similarity index 100% rename from content/guides/testcontainers-java-replace-h2/_index.md rename to content/guides/testcontainers-java-replace-h2/index.md diff --git a/content/guides/testcontainers-java-service-configuration/_index.md b/content/guides/testcontainers-java-service-configuration/index.md similarity index 100% rename from content/guides/testcontainers-java-service-configuration/_index.md rename to content/guides/testcontainers-java-service-configuration/index.md diff --git a/content/guides/testcontainers-java-spring-boot-kafka/_index.md b/content/guides/testcontainers-java-spring-boot-kafka/index.md similarity index 100% rename from content/guides/testcontainers-java-spring-boot-kafka/_index.md rename to content/guides/testcontainers-java-spring-boot-kafka/index.md diff --git a/content/guides/testcontainers-java-spring-boot-rest-api/_index.md b/content/guides/testcontainers-java-spring-boot-rest-api/index.md similarity index 100% rename from content/guides/testcontainers-java-spring-boot-rest-api/_index.md rename to content/guides/testcontainers-java-spring-boot-rest-api/index.md diff --git a/content/guides/testcontainers-java-wiremock/_index.md b/content/guides/testcontainers-java-wiremock/index.md similarity index 100% rename from content/guides/testcontainers-java-wiremock/_index.md rename to content/guides/testcontainers-java-wiremock/index.md diff --git a/content/guides/testcontainers-nodejs-getting-started/_index.md b/content/guides/testcontainers-nodejs-getting-started/index.md similarity index 100% rename from content/guides/testcontainers-nodejs-getting-started/_index.md rename to content/guides/testcontainers-nodejs-getting-started/index.md diff --git a/content/guides/testcontainers-python-getting-started/_index.md b/content/guides/testcontainers-python-getting-started/index.md similarity index 100% rename from content/guides/testcontainers-python-getting-started/_index.md rename to content/guides/testcontainers-python-getting-started/index.md diff --git a/content/guides/vuejs/_index.md b/content/guides/vuejs/index.md similarity index 100% rename from content/guides/vuejs/_index.md rename to content/guides/vuejs/index.md diff --git a/layouts/guides/landing.html b/layouts/guides/landing.html index 8ca413f5964a..fcd528d8f523 100644 --- a/layouts/guides/landing.html +++ b/layouts/guides/landing.html @@ -29,7 +29,7 @@

    - {{- len .Pages }} guides + {{- len .RegularPagesRecursive }} guides

    {{- range $i, $tag := $tagOrder }} {{- $tagData := index hugo.Data.tags $tag }} - {{- $pages := where $.Pages "Params.tags" "intersect" (slice $tag) }} + {{- $pages := where $.RegularPagesRecursive "Params.tags" "intersect" (slice $tag) }} {{- if $pages }}

    - of {{ len .Pages }} guides + of {{ len .RegularPagesRecursive }} guides

    diff --git a/layouts/guides/list.html b/layouts/guides/list.html deleted file mode 100644 index 5d7ab8ed1ff3..000000000000 --- a/layouts/guides/list.html +++ /dev/null @@ -1,14 +0,0 @@ -{{ define "left" }} - {{- partial "sidebar/mainnav.html" . }} -{{ end }} - -{{ define "main" }} -
    -
    - {{ partial "content-default.html" . }} -
    - -
    -{{ end }} diff --git a/layouts/redirect/single.html b/layouts/redirect/single.html deleted file mode 100644 index 03bf721b353b..000000000000 --- a/layouts/redirect/single.html +++ /dev/null @@ -1 +0,0 @@ -{{- template "alias.html" (dict "Permalink" .Params.target) -}} From eb68604a8e4c0c5b04fc2363f9af2abd19a152df Mon Sep 17 00:00:00 2001 From: David Karlsson <35727626+dvdksn@users.noreply.github.com> Date: Fri, 12 Jun 2026 14:09:47 +0200 Subject: [PATCH 8/8] guides: fix CI failures from the revamp (markdownlint + htmltest) - genai-video-bot, pre-seeding: remove stray "---" lines that a Phase 1 front-matter delimiter fix mistakenly inserted after lines ending in dashes (the .env separator and a psql table rule). Those lines broke out of indented code fences, so markdownlint saw code as headings (MD022/MD023/MD025) and a dangling fence (MD040). - cpp: repoint a stale body link from the old /guides/language/cpp/ multistage/ path (collapsed away) to /guides/cpp/, fixing the htmltest broken-link error. Front-matter aliases keep the old paths intact. Verified: markdownlint clean on both files; hugo --gc --minify --panicOnWarning --printUnusedTemplates builds clean. Co-Authored-By: Claude Opus 4.8 (1M context) --- content/guides/cpp/index.md | 2 +- content/guides/genai-video-bot/index.md | 4 ---- content/guides/pre-seeding.md | 4 ---- 3 files changed, 1 insertion(+), 9 deletions(-) diff --git a/content/guides/cpp/index.md b/content/guides/cpp/index.md index 8cb478ea36f0..3c5ee67cbb7c 100644 --- a/content/guides/cpp/index.md +++ b/content/guides/cpp/index.md @@ -579,7 +579,7 @@ This section walks you through extracting Software Bill of Materials (SBOMs) fro ### Generate an SBOM -Here we will use the Docker image that we built in the [Create a multi-stage build for your C++ application](/guides/language/cpp/multistage/) guide. If you haven't already built the image, follow the steps in that guide to build the image. +Here we will use the Docker image that we built in the [Create a multi-stage build for your C++ application](/guides/cpp/) guide. If you haven't already built the image, follow the steps in that guide to build the image. The image is named `hello`. To generate an SBOM for the `hello` image, run the following command: ```bash diff --git a/content/guides/genai-video-bot/index.md b/content/guides/genai-video-bot/index.md index 7a7dd5f30e0b..c8559d31eb2a 100644 --- a/content/guides/genai-video-bot/index.md +++ b/content/guides/genai-video-bot/index.md @@ -79,17 +79,13 @@ addition, it provides timestamps from the video that can help you find the sourc ```text #------------------------------------------------------------------------- ---- # OpenAI #------------------------------------------------------------------------- ---- OPENAI_TOKEN=your-api-key # Replace your-api-key with your personal API key #------------------------------------------------------------------------- ---- # Pinecone #------------------------------------------------------------------------- ---- PINECONE_TOKEN=your-api-key # Replace your-api-key with your personal API key ``` diff --git a/content/guides/pre-seeding.md b/content/guides/pre-seeding.md index 7041f7a31300..ac02b11f4b3b 100644 --- a/content/guides/pre-seeding.md +++ b/content/guides/pre-seeding.md @@ -124,7 +124,6 @@ Assuming that you have an existing Postgres database instance up and running, fo List of databases Name | Owner | Encoding | Collate | Ctype | ICU Locale | Locale Provider | Access privileges -----------+----------+----------+------------+------------+------------+-----------------+-------------------- ---- postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 | | libc | sampledb | postgres | UTF8 | en_US.utf8 | en_US.utf8 | | libc | template0 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | | libc | =c/postgres + @@ -140,7 +139,6 @@ Assuming that you have an existing Postgres database instance up and running, fo sampledb=# SELECT * FROM users; id | name | email ----+-------+---------------- ---- 1 | Alpha | alpha@example.com 2 | Beta | beta@example.com 3 | Gamma | gamma@example.com @@ -234,7 +232,6 @@ $ docker container stop postgres sampledb=# SELECT * FROM users; id | name | email ----+-------+---------------- ---- 1 | Alpha | alpha@example.com 2 | Beta | beta@example.com 3 | Gamma | gamma@example.com @@ -334,7 +331,6 @@ It is called at the end of the script to initiate the seeding process. The try.. sampledb=# SELECT * FROM todos; id | task | completed ----+----------------+-------- ---- 1 | Watch netflix | f 2 | Finish podcast | f 3 | Pick up kid | f