[Documentation]: Recommended way to handle yarn and docker

[neophob]

Goal of this issue is to update the readme file how to use yarn and docker.

I want a docker image to build my node projects - and use yarn to install the packages as the installation is much faster (and more deterministic) than using npm.

One reason why yarn is fast is of course the local yarn cache. So the docker image needs to mount the yarn cache directory when building the projects. Any other hints how to use docker and yarn?

[Daniel15]

For actually installing Yarn in a Docker or Vagrant image, you can use the Debian package repo (assuming you're using a Ubuntu or Debian Docker image). Enabling the package repo then doing apt-get install yarn will also install Node.js as a dependency.

As for mounting the cache, that's a pretty good idea, I'm not too sure how to do it though (I'm not very familiar with Docker myself).

[kyteague]

You wouldn't mount the Yarn cache directory. Instead, you should make sure you take advantage of Docker's image layer caching.

These are the commands I am using:

COPY package.json yarn.lock ./
RUN yarn --pure-lockfile

[ChristopherBiscardi]

It would depend on the environment and approach you want to take to building your assets.

repeated docker build

@kyteague's approach is one you could take if you didn't want to use a global cache and instead just cache the project's dependencies in a higher docker layer. (ie: if your development is going to be running docker build over and over without changing dependencies). If you change the package.json, you lose the cache at the higher layer and have to do a full reinstall.

manual docker run

A more sophisticated approach for development is to run a development container (node:6 or similar with yarn installed) and mount the cache in to do the install. note that the following uses a .docker-yarn-cache intended to be used with docker because libs like node-sass can have c-lib issues if you install them on OSX and then try to use them on Debian, etc. Something like:

docker run -itv ~/.docker-yarn-cache:/root/yarn-cache -v `pwd`:/opt/project --workdir /opt/project node:6 yarn

Typically I combine the docker run approach with some bash and volume mount caches in. Then at the end I copy my assets out of the container and ship them to S3, etc or COPY them into my production docker image.

yarn on host

You could also yarn on a host if it's similar to your container OS (debian/debian for example) and then write your Dockerfile to COPY the node_modules folder in with the rest of the project. This would allow you to have access to you host's .yarn-cache for speed and then you don't have to deal with installing in the docker build.

docker-compose (development)

If your project has a "watch mode" script, you can use a docker-compose file to alleviate some of the concerns of the "repeated docker build" approach by running something along the lines of yarn && yarn run watch as the command with the same volume mounts as the "manual docker run" approach.

---

So really it depends on your goals and build environment ("watch" development with Docker for Mac/CI from a dev image to an alpine-based prod image/etc).

[Daniel15]

@kyteague and @ChristopherBiscardi, great comments! I think it would be valuable to add a page to the documentation around best practices for using Yarn in Docker. Would you like to write a page about "Using Yarn in Docker" for our documentation? The website is in a separate repo: https://github.com/yarnpkg/website

[daveisfera]

Ideally, docker could use an external cache but there's some resistance to that ( moby/moby#17745 ), so adding a --no-cache option would be good so the docker image isn't made needlessly large by a cache that won't ever be used.

[tachuris]

This is the fastest setup I've get so far as yarn-cache can be reused by many containers:

Dockerfile

FROM node:6.7.0

RUN curl -o- -L https://yarnpkg.com/install.sh | bash

RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

ARG NODE_ENV
ENV NODE_ENV $NODE_ENV

COPY . /usr/src/app

EXPOSE 8000

ENTRYPOINT ["sh", "./entrypoint.sh"]

CMD ["node", "./server"]

entrypoint.sh

$HOME/.yarn/bin/yarn install --pure-lockfile
exec "$@"

docker-compose.yml

app:
  build: .
  volumes_from:
    - yarn-cache

yarn-cache:
  image: busybox
  volumes:
    - /root/.yarn-cache

[daveisfera]

@rstuven That will do the yarn install at run time rather than build time and means that the docker image is not self contained/fully reproducible (which is the reason why we and I believe many others use docker).

[tachuris]

@daveisfera Yes, that's why I stressed on its "fastest" quality. I missed to point out this is rather for development workflow where fast iterations matter most. On the other hand, the yarn.lock file should guarantee the reproducible aspect, but yes, it's not enough.

[tachuris]

Another approach: https://medium.com/@mfornasa/using-yarn-with-docker-c116ad289d56

[jeffijoe]

Am I wrong to assume that Yarn's global cache stores the package zips (which contain cross-platform code by which I mean it will be compiled at install-time)?

If the global cache only contains the downloaded archives and no build artifacts, would we not be able to at least mount the host's global cache so that the Docker container wouldn't have to download them?

[elaijuh]

@kyteague --pure-lockfile will not generate yarn.lock which means if I have changed package.json and rebuild the image, the old yarn.lock will be copied into image and not sync with the modified package.json?

[zoidyzoidzoid]

We use rocker which allows build-time mounting of a yarn-cache with a custom MOUNT command that doesn't commit the cache to the final Docker image, while using the smart Docker build layer caching as well.