Skip to content

juno-fx/juno_k3s

Repository files navigation

juno_k3s

<generator object sync_do_map at 0x7f4bf3ab8340>

Table of content


Role variables

Name Default value Description
argocd_image_pull_secret False When set, the passed in credentials will be used to pull the ArgoCD images. Example: argocd_image_pull_secret: username: my-username password: my-password registry: my-registry.com
argocd_image_pull_secret_name argo-image-pull-secret The name of the image pull secret to use for ArgoCD.
argocd_image_substitutions {'quay.io/argoproj/argocd': {'newName': 'quay.io/argoproj/argocd', 'newTag': 'v3.0.9'}, 'ghcr.io/dexidp/dex': {'newName': 'ghcr.io/dexidp/dex', 'newTag': 'v2.41.1'}, 'docker.io/library/redis': {'newName': 'docker.io/library/redis', 'newTag': '7.2.7-alpine'}}
juno_bootstrap_chart_repo_revision main The revision of the Juno-Bootstrap repository to use. This can be a branch name, tag or commit hash.
juno_bootstrap_chart_values {'genesis': {'url': '{{juno_genesis_deployment_git_url}}'}, 'ingress': {'url': 'https://kubernetes.github.io/ingress-nginx'}} Values to pass to the Juno Bootstrap chart. See: https://github.com/juno-fx/Juno-Bootstrap If you do not use a direct OCI proxy and leverage the k3s_registries_yaml var, you also could need to adjust the repository from which to pull images. For details, see: https://github.com/juno-fx/Juno-Bootstrap and the example airgapped playbook.
juno_bootstrap_git_password {{ juno_git_password }} This authenticates only the Juno-Bootstrap repository. You can leave it unchanged if both Juno-Bootstrap and Genesis-Deployment are accessible via juno_git_username&juno_git_password.You only need to set this if you use a private fork of the Juno Bootstrap repository and are not passing it in via local checkout
juno_bootstrap_git_username {{ juno_git_username }} This authenticates only the Juno-Bootstrap repository. You can leave it unchanged if both Juno-Bootstrap and Genesis-Deployment are accessible via juno_git_username&juno_git_password. You only need to set this if you use a private fork of the Juno Bootstrap repository and are not passing it in via local checkout.
juno_bootstrap_url https://github.com/juno-fx/Juno-Bootstrap.git The URL of the Juno-Bootstrap git repository. This only needs to be adjusted if you forked it or are using an airgapped environment. This can be also be a local filepath to the chart directory on the machine where ansible runs on, starting with file://
juno_genesis_deployment_git_password {{ juno_git_password }} This authenticates only the Juno Genesis Deployment repository. You can leave it unchanged if both Juno-Bootstrap and Genesis-Deployment are accessible via juno_git_username&juno_git_password.
juno_genesis_deployment_git_url https://github.com/juno-fx/Genesis-Deployment.git The URL of the Genesis-Deployment repository. Note you still need to set the juno_bootstrap_chart_values.genesis.url value to point to the Genesis-Deployment repository. This argument is only used to create the git secret. It can be left empty on a default, non-airgapped install.
juno_genesis_deployment_git_username {{ juno_git_username }} This authenticates only the Juno Genesis Deployment repository. You can leave it unchanged if both Juno-Bootstrap and Genesis-Deployment are accessible via juno_git_username&juno_git_password.
juno_git_password False The password used to authenticate with all Juno repositories you specified. If left to the default (false), a public repository is assumed.
juno_git_username oauth2 The username used to authenticate with all Juno repositories you specified. This is needed when you use a private fork of the Juno Bootstrap repository. It is particularly useful in airgapped environments, where you might neither have access to the public version and might require authentication on your Git host.
juno_install True Bootstrap Juno's Orion using https://github.com/juno-fx/Juno-Bootstrap
juno_supported_alma ['10'] The major versions of AlmaLinux Juno supported running on
juno_supported_debian ['12', '13'] The major versions of Debian Juno supported running on
juno_supported_rhel ['9', '10'] The major versions of Redhat (and Rocky) Juno supported running on
k3s_airgap_install False If true, the playbook will perform an airgapped install. Make sure all the URLs above are set to file:// or point to a local mirror. When setting the URLs to file://, they will be copied from your Ansible control host to the remote hosts.
k3s_binary_url https://github.com/k3s-io/k3s/releases/download/v1.33.1%2Bk3s1/k3s URL for the k3s binary. Can be http://, https:// OR file:// When using file://, a path from your ansible control host (where your run the playbook from) will be used. The files will be copied to the remote kubernetes hosts. This is useful for airgap installs.
k3s_bootstrap_node False The node used to bootstrap the cluster. This should only ever be a single node in your inventory! The playbook example we provide discovers this dynamically, but you can also set it manually.
k3s_bootstrap_node_ip False The IP address of an existing controlplane node, used to join the cluster. In most cases, we can automatically discover this, check out the playbook example - it does that out of the box!
k3s_clusterjoin_address False The address of the cluster to join. Can only be false when k3s_bootstrap_node is true.
k3s_control_plane_node False When true, join the node to an existing cluster as a control plane node. When neither k3s_bootstrap_node nor k3s_control_plane_node is true, the node will be a worker node.
k3s_copy_images {{ k3s_airgap_install and not k3s_registries_yaml }} If true, the role will copy the k3s images tarball to the standard location where k3s can load them. By default, we don't perform this if you define registries.yaml, as it is assumed you will have a local mirror.
k3s_force_reinstall False If true, rerun the k3s install script even if the node is already part of a cluster.
k3s_image_folder /var/lib/rancher
k3s_images_url https://github.com/k3s-io/k3s/releases/download/v1.33.1%2Bk3s1/k3s-airgap-images-amd64.tar.gz URL for the k3s images tarball. Can be http://, https:// OR file:// When using file://, a path from your ansible control host (where your run the playbook from) will be used. The files will be copied to the remote kubernetes hosts. This is useful for airgap installs.
k3s_install_script_url https://get.k3s.io/ URL for the k3s install script. Can be http://, https:// OR file:// When using file://, a path from your ansible control host (where your run the playbook from) will be used. The files will be copied to the remote kubernetes hosts. This is useful for airgap installs.
k3s_join_token The token used to join the cluster. You can specify it explicitly or let the playbook autodiscover it. Check out the example playbook for how to do that.k3s_join_token: false
k3s_node_labels ["{{ k3s_control_plane_node ternary('juno-innovations.com/service=true', 'juno-innovations.com/workstation=true') }}"]
k3s_open_file_limit 1048576 The open file limit for k3s, this sets the LimitNOFILE setting
k3s_perform_node_labeling True Whether to label nodes when performing the initial k3s install. Already existing nodes will not be labeled - use kubectl instead, per: https://juno-fx.github.io/Orion-Documentation/installation/pre-reqs/requirements/?h=label#1-labeling-nodes
k3s_registries_yaml False If true, the playbook will configure the registries.yaml file to use your internal mirror. For syntax refer to https://docs.k3s.io/installation/private-registry The data you pass in here will be directly templated into the registries.yaml file.
k3s_uninstall False If true, the playbook will run the default uninstall script (/usr/local/bin/k3s-uninstall.sh) This is intended mostly for quick testing - in production, ideally you'd reprovision freshly.
validate_os_version True Check we are on a supported OS version, error otherwise.

Example playbook - online/internet-enabled installation

---
- name: Ensure the correct state of all nodes in the cluster
  hosts: all
  vars:
  tasks:
    - name: Check if the join token file exists
      ansible.builtin.stat:
        path: /var/lib/rancher/k3s/server/token
      become: true
      register: stat_k3s_bootstrap_node_token_file
    - name: If the token exists on any node, set the skip_bootstrap variable
      ansible.builtin.set_fact:
        k3s_skip_bootstrap: false
      when: stat_k3s_bootstrap_node_token_file.stat.exists
      run_once: true

    - name: Make k3s_skip_bootstrap available to all hosts
      ansible.builtin.set_fact:
        k3s_skip_bootstrap: "{{ k3s_skip_bootstrap | default(false) }}"
      loop: "{{ ansible_play_hosts }}"
      delegate_to: "{{ item }}"
      run_once: true
    
    - any_errors_fatal: true
      block:
        - name: If the join token doesn't exist at all, bootstrap the cluster on the 1st control plane node in the inventory
          vars:
            k3s_bootstrap_node: true
          ansible.builtin.include_role:
            name: "juno-fx.juno_k3s"
          when:
            - inventory_hostname == (
                ansible_play_hosts_all
                | map('extract', hostvars)
                | selectattr('k3s_control_plane_node', 'defined')
                | selectattr('k3s_control_plane_node', 'equalto', true)
                | map(attribute='inventory_hostname')
                | list
                | first
              )
            - not stat_k3s_bootstrap_node_token_file.stat.exists
            - k3s_control_plane_node | default(false) | bool
            - not k3s_skip_bootstrap | default(false)


    - name: Check if the join token file exists (again)
      ansible.builtin.stat:
        path: /var/lib/rancher/k3s/server/token
      become: true
      register: stat_k3s_bootstrap_node_token_file

    - name: If the join token file exists across any of the control plane nodes, set the variable
      become: true
      ansible.builtin.slurp:
        src: /var/lib/rancher/k3s/server/token
      when: stat_k3s_bootstrap_node_token_file.stat.exists
      register: slurp_k3s_bootstrap_node_token_file
      run_once: true

    - name: Make the variable available to the play
      ansible.builtin.set_fact:
        k3s_bootstrap_node_token: "{{ slurp_k3s_bootstrap_node_token_file.content | b64decode }}"
        k3s_bootstrap_node_ip: "{{ ansible_default_ipv4.address }}"
      when: stat_k3s_bootstrap_node_token_file.stat.exists
      loop: "{{ ansible_play_hosts }}"
      run_once: true
      delegate_to: "{{ item }}"
    - name: Gather service facts on the control plane nodes
      ansible.builtin.service_facts:
      when: k3s_control_plane_node | default(false) | bool

    - name: Ensure k3s control plane nodes
      vars:
        k3s_join_token: "{{ k3s_bootstrap_node_token }}"
      when:
        - k3s_control_plane_node | default(false) | bool
      ansible.builtin.include_role:
        name: "juno-fx.juno_k3s"

    - name: Ensure k3s worker nodes
      vars:
        k3s_join_token: "{{ k3s_bootstrap_node_token }}"
      when:
        - not k3s_control_plane_node | default(false) | bool
      ansible.builtin.include_role:
        name: "juno-fx.juno_k3s"

Example playbook - airgapped installation

The below playbook assumes you have downloaded all the necessary files listed in vars: and passed in the paths to them. You can check the detailed information for each file in the vars section above.

---

- name: Ensure the correct state of all nodes in the cluster
  hosts:
    - control_plane
    - k8s_worker
  vars:
    juno_git_user: "oauth2"
    juno_git_password: "password"
    juno_genesis_deployment_git_url: "http://{{ proxy_address }}/git/Genesis-Deployment.git"
    nginx_ingress_deployment_git_url: "http://{{ proxy_address }}/git/ingress-nginx.git"
    juno_bootstrap_url: "http://{{ proxy_address }}/git/Juno-Bootstrap.git"
    k3s_install_script_url: "file://{{ playbook_dir }}/airgap_files/install.sh"
    k3s_binary_url: "file://{{ playbook_dir }}/airgap_files/k3s"
    # For more details on using a private registry, eg. using authentication, see:
    # https://docs.k3s.io/installation/private-registry
    k3s_registries_yaml: |
      mirrors:
        docker.io:
          endpoint:
            - "http://{{ proxy_address }}:5000"
        quay.io:
          endpoint:
            - "http://{{ proxy_address }}:5001"
        ghcr.io:
          endpoint:
            - "http://{{ proxy_address }}:5002"
    k3s_airgap_install: true
    juno_bootstrap_chart_values:
      genesis:
        url: "{{ juno_genesis_deployment_git_url }}"
      ingress:
        url: "{{ nginx_ingress_deployment_git_url }}"
  tasks:
    - name: Check if the join token file exists
      ansible.builtin.stat:
        path: /var/lib/rancher/k3s/server/token
      become: true
      register: stat_k3s_bootstrap_node_token_file
    - name: If the token exists on any node, set the skip_bootstrap variable
      ansible.builtin.set_fact:
        k3s_skip_bootstrap: false
      when: stat_k3s_bootstrap_node_token_file.stat.exists
      run_once: true

    - name: Make k3s_skip_bootstrap available to all hosts
      ansible.builtin.set_fact:
        k3s_skip_bootstrap: "{{ k3s_skip_bootstrap | default(false) }}"
      loop: "{{ ansible_play_hosts }}"
      delegate_to: "{{ item }}"
      run_once: true

    - any_errors_fatal: true
      block:
        - name: If the join token doesn't exist at all, bootstrap the cluster on the 1st control plane node in the inventory
          vars:
            k3s_bootstrap_node: true
          ansible.builtin.include_role:
            name: "juno-fx.juno_k3s"
          when:
            - inventory_hostname == (
                ansible_play_hosts_all
                | map('extract', hostvars)
                | selectattr('k3s_control_plane_node', 'defined')
                | selectattr('k3s_control_plane_node', 'equalto', true)
                | map(attribute='inventory_hostname')
                | list
                | first
              )
            - not stat_k3s_bootstrap_node_token_file.stat.exists
            - k3s_control_plane_node | default(false) | bool
            - not k3s_skip_bootstrap | default(false)

    - name: Check if the join token file exists (again)
      ansible.builtin.stat:
        path: /var/lib/rancher/k3s/server/token
      become: true
      register: stat_k3s_bootstrap_node_token_file

    - name: If the join token file exists across any of the control plane nodes, set the variable
      become: true
      ansible.builtin.slurp:
        src: /var/lib/rancher/k3s/server/token
      when: stat_k3s_bootstrap_node_token_file.stat.exists
      register: slurp_k3s_bootstrap_node_token_file
      run_once: true

    - name: Make the variable available to the play
      ansible.builtin.set_fact:
        k3s_bootstrap_node_token: "{{ slurp_k3s_bootstrap_node_token_file.content | b64decode }}"
        k3s_bootstrap_node_ip: "{{ ansible_default_ipv4.address }}"
      when: stat_k3s_bootstrap_node_token_file.stat.exists
      loop: "{{ ansible_play_hosts }}"
      run_once: true
      delegate_to: "{{ item }}"
    - name: Gather service facts on the control plane nodes
      ansible.builtin.service_facts:
      when: k3s_control_plane_node | default(false) | bool

    - name: Ensure k3s control plane nodes
      vars:
        k3s_join_token: "{{ k3s_bootstrap_node_token }}"
      when:
        - k3s_control_plane_node | default(false) | bool
      ansible.builtin.include_role:
        name: "juno-fx.juno_k3s"

    - name: Ensure k3s worker nodes
      vars:
        k3s_join_token: "{{ k3s_bootstrap_node_token }}"
      when:
        - not k3s_control_plane_node | default(false) | bool
      ansible.builtin.include_role:
        name: "juno-fx.juno_k3s"

Dependencies

None.

License

Apache-2.0

Author

Juno Innovations

Development workflow

This repository comes in with a Makefile providing targets for testing & linting the role.

For usage examples see: CONTRIBUTING.md

Supported Operating Systems

The following major versions of the specified Linux distributions are supported by Juno

Debian

  • 12
  • 13

Redhat and Rocky

  • 9
  • 10

AlmaLinux

  • 10

About

Ansible role for Orion by Juno Innovations & the underlying k3s cluster (batteries included!)

Resources

Contributing

Stars

Watchers

Forks

Contributors