Poetry + Docker

Docker FTW <3

Author: pythoninthegrass

.dockerignore

@@ -10,14 +10,17 @@
 **/.project
 **/.settings
 **/.toolstarget
+**/.venv
 **/.vs
 **/.vscode
 **/*.*proj.user
 **/*.dbmdl
+**/*.example
 **/*.jfm
 **/*.py#
 **/*.py~
 **/*.pyc
+**/*test.*
 **/azds.yaml
 **/bin
 **/charts
@@ -26,11 +29,15 @@
 **/django
 **/docker-compose*
 **/Dockerfile*
+**/heroku.yml
 **/img
 **/node_modules
 **/npm-debug.log
 **/obj
+**/pre-commit-config.yaml
+**/Procfile
 **/README.md
+**/runtime.txt
 **/secrets.dev.yaml
 **/terraform
 **/values.dev.yaml

.env.example

@@ -0,0 +1,2 @@
+JUPYTER_TOKEN=docker
+PORT=8888

.gitignore

@@ -3,11 +3,8 @@ django/composeexample/
 django/data/
 django/manage.py
 *.sqlite
-redis*
-!redis.conf.example
 *.pem
 *.key
-raw/
 gitleaks_report*.json
 
 # Byte-compiled / optimized / DLL files

.pre-commit-config.yaml

@@ -4,11 +4,11 @@ verbose: true
 repos:
   # checks for secrets via rules (gitleaks.toml)
   - repo: https://github.com/zricethezav/gitleaks
-    rev: v8.9.0
+    rev: v8.13.0
     hooks:
       - id: gitleaks
   - repo: https://github.com/ambv/black
-    rev: 22.6.0
+    rev: 22.8.0
     hooks:
     - id: black
   - repo: https://gitlab.com/pycqa/flake8

Dockerfile

@@ -1,111 +1,26 @@
-# using ubuntu LTS version
-FROM ubuntu:22.04 AS builder-image
-
-# avoid stuck build due to user prompt
-ARG DEBIAN_FRONTEND=noninteractive
-
-RUN apt-get -qq update \
-    && apt-get -qq install \
-    --no-install-recommends -y \
-    aptitude \
-    autoconf \
-    automake \
-    build-essential \
-    ca-certificates \
-    curl \
-    git \
-    locales \
-    libbz2-dev \
-    libffi-dev \
-    libncurses-dev \
-    libreadline-dev \
-    libsqlite3-dev \
-    libssl-dev \
-    libtool \
-    libxslt-dev \
-    libyaml-dev \
-    python3 \
-    python3-dev \
-    python3-pip \
-    sqlite3 \
-    unixodbc-dev \
-    unzip \
-    && rm -rf /var/lib/apt/lists/*
-
-# Set locale
-RUN locale-gen en_US.UTF-8
-ENV LANG=en_US.UTF-8
-ENV LANGUAGE=en_US:en
-ENV LC_ALL=en_US.UTF-8
-
-ARG USERNAME=appuser
-ENV HOME="/home/${USERNAME}"
-ENV PATH="$HOME/.asdf/bin:$HOME/.asdf/shims:$PATH"
-
-RUN useradd --create-home $USERNAME
-
-# install asdf then python latest
-RUN bash -c "git clone --depth 1 https://github.com/asdf-vm/asdf.git $HOME/.asdf \
-    && echo '. $HOME/.asdf/asdf.sh' >> $HOME/.bashrc  \
-    && echo '. $HOME/.asdf/asdf.sh' >> $HOME/.profile"
-RUN asdf plugin-add python \
-    && asdf install python 3.10.7 \
-    && asdf global python 3.10.7
-
-# TODO: test poetry via asdf
-ENV POETRY_HOME="$HOME/.poetry"
-RUN curl -sSL https://install.python-poetry.org | python3.10 -
-ENV PATH "${POETRY_HOME}/bin:$PATH"
-
-WORKDIR $$HOME/app
-COPY pyproject.toml poetry.lock ./
-RUN python3.10 -m venv /opt/venv
-
-# Install pip requirements
-RUN . /opt/venv/bin/activate && poetry install
-
-# TODO: dive + docker-slim
-FROM ubuntu:22.04 AS runner-image
-
-ARG USERNAME=appuser
-ENV HOME="/home/${USERNAME}"
-ENV VIRTUAL_ENV="/opt/venv"
-ENV PATH="${VIRTUAL_ENV}/bin:$HOME/.asdf/bin:$HOME/.asdf/shims:$PATH"
-
-RUN useradd --create-home $USERNAME \
-    && mkdir -p ${HOME}/app
-
-COPY --chown=${USERNAME}:${USERNAME} . $HOME/app
-COPY --from=builder-image --chown=${USERNAME}:${USERNAME} /opt/venv /opt/venv
-COPY --from=builder-image --chown=${USERNAME}:${USERNAME} $HOME/.asdf $HOME/.asdf
-
-# avoid stuck build due to user prompt
-ARG DEBIAN_FRONTEND=noninteractive
-
-RUN apt-get -qq update \
-    && apt-get -qq install \
-    --no-install-recommends -y \
-    ca-certificates \
-    curl \
-    git \
-    libsqlite3-dev \
-    sqlite3 \
-    && rm -rf /var/lib/apt/lists/*
+FROM jupyter/minimal-notebook:latest
 
 # Keeps Python from generating .pyc files in the container
 ENV PYTHONDONTWRITEBYTECODE=1
 
 # Turns off buffering for easier container logging
 ENV PYTHONUNBUFFERED=1
 
-# activate virtual environment
-RUN python -m venv $VIRTUAL_ENV
+USER jovyan
+
+COPY . $HOME/work
 
-USER appuser
+WORKDIR $HOME/work
 
-WORKDIR $HOME/app
+RUN conda install \
+    xarray \
+    netCDF4 \
+    bottleneck \
+    numpy \
+    pandas \
+    matplotlib \
+    jupyterlab
 
 # ENTRYPOINT ["python", "main.py"]
-# CMD ["gunicorn", "-c", "config/gunicorn.conf.py", "main:app"]
 CMD ["/bin/bash", "startup.sh"]
 # CMD ["/bin/bash"]

README.md

@@ -1,57 +1,76 @@
-# python_template
+# wine_quality
 
-!["It's dangerous to go alone! Take this."](img/zelda.jpg)
-<!-- <img src="https://user-images.githubusercontent.com/4097471/144654508-823c6e31-5e10-404c-9f9f-0d6b9d6ce617.jpg" width="300"> -->
+![JupyterLab](img/jupyterlab.png)
 
 ## Summary
-Oftentimes the initial setup of a Python repo can take a few minutes to a couple hours.
-By laying the foundation to rapidly implement an idea, can focus on the good bits instead of
-devops drudgery.
-
-### Caveat Emptor
-Very little of this gets tested on Windows hosts. Windows Subsystem for Linux (WSL) is used where necessary with the default Ubuntu LTS install. Moved bulk of document to the [markdown](markdown/) directory to opt-in vs. opt-out of documentation.
-
-Be the change et al if Windows is your main and you wanna raise a PR with broad instructions on getting tooling working under Windows (e.g., docker, poetry, playwright.)
+Udacity course on data science.
 
 **Table of Contents**
-* [python_template](#python_template)
+* [wine_quality](#wine_quality)
   * [Summary](#summary)
-    * [Caveat Emptor](#caveat-emptor)
   * [Setup](#setup)
   * [Usage](#usage)
-    * [Mac and Linux users](#mac-and-linux-users)
+    * [Poetry](#poetry)
+    * [Docker](#docker)
+    * [Both](#both)
   * [TODO](#todo)
+  * [Further Reading](#further-reading)
 
 ## Setup
 * Install
     * [editorconfig](https://editorconfig.org/)
-    * [wsl](https://docs.microsoft.com/en-us/windows/wsl/setup/environment)
     * [asdf](https://asdf-vm.com/guide/getting-started.html#_2-download-asdf)
     * [poetry](https://python-poetry.org/docs/)
-    * [docker-compose](https://docs.docker.com/compose/install/)
-    * [playwright](https://playwright.dev/python/docs/intro#installation)
-    * [Kubernetes (k8s)](markdown/kubernetes.md)
+    * [docker](https://docs.docker.com/compose/install/)
+* Download CSVs from [here](https://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/)
+  * Move to `csv` directory
 
 ## Usage
-### Mac and Linux users
-Development environments and tooling are first-class citizens on macOS and *nix. For Windows faithfuls, please setup [WSL](markdown/wsl.md).
+### Poetry
+* Install requirements via Poetry: 
+    ```bash
+    poetry install
+    poetry run ipython kernel install --name "python3.10.7" --user
+    ```
+* Run Jupyter Lab
+    ```bash
+    poetry shell
+    jupyter lab --ip=0.0.0.0 --port=8888 --no-browser
+    ```
+* Quit the server via `ctrl-c` in the terminal
+* Enter `deactivate` to exit the Poetry virtual environment
+
+### Docker
+* Customize the `.env.example` and rename to `.env`
+* General commands
+```bash
+# build image locally
+docker-compose build --pull --no-cache
+
+# start container
+docker-compose up -d
+
+# stop container
+docker-compose stop
+
+# remove container and network
+docker-compose down
+```
+
+### Both
+* Open a browser and navigate to `http://127.0.0.1:888`
+  * Docker uses the token specified in `.env`
+* Select the "python3.10.7" kernel if asked
+* Open `refactor-wine-quality.ipynb` from the left-hand column
+* Run cells by selecting them and pressing `shift-enter`
 
 ## TODO
-* Django
-    * Merge with [docker_python](https://github.com/pythoninthegrass/docker_python) and put the latter on an ice float
-* Flask
-    * Bonus points for [Svelte](https://svelte.dev/blog/the-easiest-way-to-get-started) front-end ❤️
-* FastAPI
-* MongoDB
-  * Switch to `docker-compose`
-  * Fix unique index deleting too many keys
-* k8s
-  * `~/.kubeconfig`
-* ansible
-* wsl
-  * VSCode
-      * Remote WSL install and usage
-        * Or at least further reading nods
-* Debugging
-   * Dependencies
-   * script itself via [icecream](https://github.com/gruns/icecream)
+* Get Jupyter working in [VSCode](https://github.com/microsoft/vscode-jupyter)
+* `Makefile`
+
+## Further Reading
+[Starting JupyterLab](https://jupyterlab.readthedocs.io/en/stable/getting_started/starting.html)
+
+[Jupyter Docker Stacks — Docker Stacks documentation](https://jupyter-docker-stacks.readthedocs.io/en/latest/index.html)
+
+[Dockerizing Jupyter Projects](https://towardsdatascience.com/dockerizing-jupyter-projects-39aad547484a)

compose.yml

@@ -0,0 +1,20 @@
+version: "3.9"
+
+services:
+  jupyter:
+    image: jupyter/minimal-notebook:latest
+    tty: true               # false for `entrypoint` in Dockerfile
+    stdin_open: true        # false for `entrypoint` in Dockerfile
+    env_file:
+      - ./.env
+    volumes:
+      - .:/home/jovyan/work
+    ports:
+      - 8888:${PORT:-8888}
+    build:
+      context: ./
+      dockerfile: ./Dockerfile
+
+networks:
+  default:
+    driver: bridge