31 lines (26 loc) · 716 Bytes

DataTalks Data Engineering Zoomcamp

Covered Topics

Docker

How to write a Dockerfile
Building image using Dockerfile
Running the image
Running two docker containers in same network
Port mapping
Docker Compose

Terraform

Using Infrastructure as code
Creating terraform config
Planning, applying and destroying over GCP

GCP

Setup Bigquery, Storage using Terraform
Setup VM instance
Create SSH config file
SSH into the VM instance from local
Access code dir in VScode
Map GCP ports to local instance

Postgres, PgAdmin, SQL

Setup postgres and pgadmin in docker
Insert data into postgres using pandas.io.sql
connect using pgcli
SQL refresher

Big Query