Skip to content

09-docker-compose.md

Latest commit

 

History

History
106 lines (77 loc) · 3 KB

09-docker-compose.md

File metadata and controls

106 lines (77 loc) · 3 KB

Docker Compose

↑ Up | ← Previous | Next →

docker-compose allows us to launch multiple containers using a single configuration file, so that we don't have to run multiple complex docker run commands separately.

Docker compose makes use of YAML files. Here's the docker-compose.yaml file:

services:
  pgdatabase:
    image: postgres:18
    environment:
      POSTGRES_USER: "root"
      POSTGRES_PASSWORD: "root"
      POSTGRES_DB: "ny_taxi"
    volumes:
      - "ny_taxi_postgres_data:/var/lib/postgresql"
    ports:
      - "5432:5432"

  pgadmin:
    image: dpage/pgadmin4
    environment:
      PGADMIN_DEFAULT_EMAIL: "admin@admin.com"
      PGADMIN_DEFAULT_PASSWORD: "root"
    volumes:
      - "pgadmin_data:/var/lib/pgadmin"
    ports:
      - "8085:80"



volumes:
  ny_taxi_postgres_data:
  pgadmin_data:

Explanation

  • We don't have to specify a network because docker compose takes care of it: every single container (or "service", as the file states) will run within the same network and will be able to find each other according to their names (pgdatabase and pgadmin in this example).
  • All other details from the docker run commands (environment variables, volumes and ports) are mentioned accordingly in the file following YAML syntax.

Start Services with Docker Compose

We can now run Docker compose by running the following command from the same directory where docker-compose.yaml is found. Make sure that all previous containers aren't running anymore:

docker-compose up

Detached Mode

If you want to run the containers again in the background rather than in the foreground (thus freeing up your terminal), you can run them in detached mode:

docker-compose up -d

Stop Services

You will have to press Ctrl+C in order to shut down the containers when running in foreground mode. The proper way of shutting them down is with this command:

docker-compose down

Other Useful Commands

# View logs
docker-compose logs

# Stop and remove volumes
docker-compose down -v

Benefits of Docker Compose

  • Single command to start all services
  • Automatic network creation
  • Easy configuration management
  • Declarative infrastructure

Running the Ingestion Script with Docker Compose

If you want to re-run the dockerized ingest script when you run Postgres and pgAdmin with docker compose, you will have to find the name of the virtual network that Docker compose created for the containers.

# check the network link:
docker network ls

# it's pipeline_default (or similar based on directory name)
# now run the script:
docker run -it --rm\
  --network=pipeline_default \
  taxi_ingest:v001 \
    --pg-user=root \
    --pg-pass=root \
    --pg-host=pgdatabase \
    --pg-port=5432 \
    --pg-db=ny_taxi \
    --target-table=yellow_taxi_trips

↑ Up | ← Previous | Next →