Write value to file

[theoephraim]

Often as part of our secrets / config, we encounter certificates or other things that are usually stored in files. Especially if they are large, passing things around through env vars doesn't really make sense. So we want a way to use varlock to fetch the value and write it to a file. The overall goal would be to make this work for any plugin, and to probably keep the "fetch the value" part within an item value, rather than a decorator.

In general here the idea of having data that is not injected as env vars may be a helpful building block. See #549 for some discussion.

A few options / ideas:

Option 1 - new file data type

Add a new data type called file which is meant to save the contents of the item to a path, and then return the file path as the env var. Conceptually it's a little weirder but is relatively simple.

# @type=file(./certs/my-cert.crt)
MY_CERT=op("op://...")

Option 2 - writeFile resolver which returns the path

Keep the value more related to being a file path, but have the data written as a side effect of a function. (Note that path is not a real type yet, but one that is planned on being added anyway).

# @type=path
MY_CERT_PATH=writeFile(./certs/my-cert.crt, op("op://...")) # writes the file and returns the path

Option 3 - writeFile resolver which returns the file contents

Break it up into 2 items, and have a writeFile function that writes the file but returns the value rather than the path.
In this case we may want another decorator to mark items that should not be injected as env vars (or could maybe use a naming convention). This one allows for validating both the path and the file contents independently, and is perhaps the most clear, but it's also a bit verbose.

We should also consider that often we wouldn't really need to inject the path as an env var because it is going to a known constant location, so you may not always need both items.

# @type=path
MY_CERT_PATH=./certs/my-cert.crt
# @type=crt @noEnvVar
MY_CERT_VALUE=writeFile($MY_CERT_PATH, op("op://..."))

Option 4 - writeFile root decorator

Could just make writeFile a decorator. Root decorators must go in the header, but we could consider allowing these unattached decorators anywhere in the file. This way you'd be writing to a file, but no env var is needed. Or you could attach to an item too...

root decorator

# @writeFile(./certs/my-cert.crt, $MY_CERT_VALUE) # using an env var
# @writeFile(./certs/my-cert2.crt, op(...)) # without an env var

# @type=crt
MY_CERT_VALUE=

item decorator

# @type=crt @writeValueToFile(./certs/my-cert.crt)
MY_CERT_VALUE=op(...)

[theoephraim]

also realized we might want a read from file function as well. Useful for things like JSON creds that GCP gives you.

[astralarya]

A quick summary of what I am doing today without varlock:

I use mkcert to create an SSL certificate: https://github.com/FiloSottile/mkcert

.env:

SSL_KEY=$HOME/cert/dev.local+4-key.pem
SSL_CERT=$HOME/cert/dev.local+4.pem
LOCAL_DOMAIN=dev.local

vite.config.ts:

import { readFileSync } from "node:fs";
import { dirname } from "node:path";
import { fileURLToPath } from "node:url";
import { loadEnv } from "vite";

const __dirname = dirname(fileURLToPath(import.meta.url));
const env = loadEnv("development", __dirname, "");

const readFile = (path: string | undefined) =>
  path ? readFileSync(path, "utf-8") : undefined

const sslKey = readFile(env.SSL_KEY);
const sslCert = readFile(env.SSL_CERT);
const localDomain = env.LOCAL_DOMAIN

/// <reference types="vitest" />

export default defineConfig({
  server: {
    https:
      sslKey && sslCert
        ? {
            key: sslKey,
            cert: sslCert,
          }
        : undefined,
    allowedHosts: localDomain ? [localDomain] : [],
  },
});