|
2 | 2 |
|
3 | 3 | This module creates a [Slinky](https://slinky.ai) cluster and nodesets, for a [Slurm](https://slurm.schedmd.com/documentation.html)-on-Kubernetes HPC setup. |
4 | 4 |
|
5 | | -The setup closely follows the [documented quickstart installation](https://github.com/SlinkyProject/slurm-operator/blob/main/docs/quickstart.md), with the exception of a more lightweight monitoring/metrics setup. Consider scraping the Slurm Exporter with [Google Managed Prometheus](https://cloud.google.com/stackdriver/docs/managed-prometheus) and a [PodMonitoring resource](https://cloud.google.com/stackdriver/docs/managed-prometheus/setup-managed#gmp-pod-monitoring), rather than a cluster-local Kube Prometheus Stack (although both are possible with module parameterizations). |
| 5 | +The setup closely follows the [documented quickstart installation v0.3.1](https://github.com/SlinkyProject/slurm-operator/blob/v0.3.1/docs/quickstart.md), with the exception of a more lightweight monitoring/metrics setup. Consider scraping the Slurm Exporter with [Google Managed Prometheus](https://cloud.google.com/stackdriver/docs/managed-prometheus) and a [PodMonitoring resource](https://cloud.google.com/stackdriver/docs/managed-prometheus/setup-managed#gmp-pod-monitoring), rather than a cluster-local Kube Prometheus Stack (although both are possible with module parameterizations). It also provisions a login node (pod). |
6 | 6 |
|
7 | 7 | Through `cert_manager_values`, `prometheus_values`, `slurm_operator_values`, and `slurm_values`, you can customize the Helm releases that constitute Slinky. The Cert Manager, Slurm Operator, and Slurm Helm installations are required, whereas the Prometheus Helm chart is optional (and not included by default). Set `install_kube_prometheus_stack=true` to install Prometheus. |
8 | 8 |
|
@@ -41,30 +41,66 @@ Through `cert_manager_values`, `prometheus_values`, `slurm_operator_values`, and |
41 | 41 | - h3-standard-88 |
42 | 42 | partition: |
43 | 43 | enabled: true |
| 44 | + login: # Login node |
| 45 | + enabled: true |
| 46 | + replicas: 1 |
| 47 | + rootSshAuthorizedKeys: [] |
| 48 | + image: |
| 49 | + # Use the default login image |
| 50 | + repository: "" |
| 51 | + tag: "" |
| 52 | + resources: |
| 53 | + requests: |
| 54 | + cpu: 500m |
| 55 | + memory: 4Gi |
| 56 | + limits: |
| 57 | + cpu: 500m |
| 58 | + memory: 4Gi |
| 59 | + affinity: |
| 60 | + nodeAffinity: |
| 61 | + requiredDuringSchedulingIgnoredDuringExecution: |
| 62 | + nodeSelectorTerms: |
| 63 | + - matchExpressions: |
| 64 | + - key: "node.kubernetes.io/instance-type" |
| 65 | + operator: In |
| 66 | + values: |
| 67 | + - e2-standard-8 # base_pool's machine-type |
44 | 68 | ``` |
45 | 69 |
|
46 | 70 | This creates a Slinky cluster with the following attributes: |
47 | 71 |
|
48 | 72 | * Slinky Helm releases are installed atop the `gke_cluster` (from the `gke-cluster` module). |
49 | | -* Slinky system components are scheduled on the `base_pool` (from the `gke-node-pool` module). |
| 73 | +* Slinky system components and a login node are scheduled on the `base_pool` (from the `gke-node-pool` module). |
50 | 74 | * This node affinity specification is recommended, to save HPC hardware for HPC nodesets, and to ensure Helm releases are fully uninstalled before all nodepools are deleted during a `gcluster destroy`. |
51 | 75 | * One Slurm nodeset is provisioned, with resource requests/limits and node affinities aligned to h3-standard-88 VMs. |
52 | 76 |
|
53 | 77 | ### Usage |
54 | 78 |
|
55 | | -To test Slurm functionality, connect to the controller and use Slurm client commands: |
| 79 | +To test Slurm functionality, connect to the controller or the login node and use Slurm client commands: |
56 | 80 |
|
57 | 81 | ```bash |
58 | 82 | gcloud container clusters get-credentials YOUR_CLUSTER --region YOUR_REGION |
59 | 83 | ``` |
60 | 84 |
|
| 85 | +Connect to the controller: |
| 86 | + |
61 | 87 | ```bash |
62 | 88 | kubectl exec -it statefulsets/slurm-controller \ |
63 | 89 | --namespace=slurm \ |
64 | 90 | -- bash --login |
65 | 91 | ``` |
66 | 92 |
|
67 | | -On the controller pod (e.g. host slurm@slurm-controller-0), run the following commands to quickly test if Slurm is functioning: |
| 93 | +Connect to the login node: |
| 94 | + |
| 95 | +```bash |
| 96 | +SLURM_LOGIN_IP="$(kubectl get services -n slurm -l app.kubernetes.io/instance=slurm,app.kubernetes.io/name=login -o jsonpath="{.items[0].status.loadBalancer.ingress[0].ip}")" |
| 97 | +## Assuming your public SSH key was configured in `login.rootSshAuthorizedKeys[]`. |
| 98 | +ssh -p 2222 root@${SLURM_LOGIN_IP} |
| 99 | +## Assuming SSSD is configured. |
| 100 | +ssh -p 2222 ${USER}@${SLURM_LOGIN_IP} |
| 101 | +``` |
| 102 | + |
| 103 | +On the connected pod (e.g. host slurm@slurm-controller-0), run the following commands to quickly test if Slurm is functioning: |
68 | 104 |
|
69 | 105 | ```bash |
70 | 106 | sinfo |
@@ -108,19 +144,19 @@ No modules. |
108 | 144 |
|
109 | 145 | | Name | Description | Type | Default | Required | |
110 | 146 | |------|-------------|------|---------|:--------:| |
111 | | -| <a name="input_cert_manager_chart_version"></a> [cert\_manager\_chart\_version](#input\_cert\_manager\_chart\_version) | Version of the Cert Manager chart to install. | `string` | `"v1.17.1"` | no | |
| 147 | +| <a name="input_cert_manager_chart_version"></a> [cert\_manager\_chart\_version](#input\_cert\_manager\_chart\_version) | Version of the Cert Manager chart to install. | `string` | `"v1.18.2"` | no | |
112 | 148 | | <a name="input_cert_manager_values"></a> [cert\_manager\_values](#input\_cert\_manager\_values) | Value overrides for the Cert Manager release | `any` | <pre>{<br/> "crds": {<br/> "enabled": true<br/> }<br/>}</pre> | no | |
113 | 149 | | < a name= "input_cluster_id"></ a> [cluster \_id ](#input\_cluster\_id) | An identifier for the GKE cluster resource with format projects/<project \_id>/locations/< region>/clusters/< name>. | `string` | n/a | yes | |
114 | 150 | | <a name="input_install_kube_prometheus_stack"></a> [install\_kube\_prometheus\_stack](#input\_install\_kube\_prometheus\_stack) | Install the Kube Prometheus Stack. | `bool` | `false` | no | |
115 | 151 | | <a name="input_install_slurm_chart"></a> [install\_slurm\_chart](#input\_install\_slurm\_chart) | Install slurm-operator chart. | `bool` | `true` | no | |
116 | 152 | | <a name="input_install_slurm_operator_chart"></a> [install\_slurm\_operator\_chart](#input\_install\_slurm\_operator\_chart) | Install slurm-operator chart. | `bool` | `true` | no | |
117 | 153 | | <a name="input_node_pool_names"></a> [node\_pool\_names](#input\_node\_pool\_names) | Names of node pools, for use in node affinities (Slinky system components). | `list(string)` | `null` | no | |
118 | 154 | | <a name="input_project_id"></a> [project\_id](#input\_project\_id) | The project ID that hosts the GKE cluster. | `string` | n/a | yes | |
119 | | -| <a name="input_prometheus_chart_version"></a> [prometheus\_chart\_version](#input\_prometheus\_chart\_version) | Version of the Kube Prometheus Stack chart to install. | `string` | `"70.4.1"` | no | |
| 155 | +| <a name="input_prometheus_chart_version"></a> [prometheus\_chart\_version](#input\_prometheus\_chart\_version) | Version of the Kube Prometheus Stack chart to install. | `string` | `"77.0.1"` | no | |
120 | 156 | | <a name="input_prometheus_values"></a> [prometheus\_values](#input\_prometheus\_values) | Value overrides for the Prometheus release | `any` | <pre>{<br/> "installCRDs": true<br/>}</pre> | no | |
121 | | -| <a name="input_slurm_chart_version"></a> [slurm\_chart\_version](#input\_slurm\_chart\_version) | Version of the Slurm chart to install. | `string` | `"0.2.1"` | no | |
| 157 | +| <a name="input_slurm_chart_version"></a> [slurm\_chart\_version](#input\_slurm\_chart\_version) | Version of the Slurm chart to install. | `string` | `"0.3.1"` | no | |
122 | 158 | | <a name="input_slurm_namespace"></a> [slurm\_namespace](#input\_slurm\_namespace) | slurm namespace for charts | `string` | `"slurm"` | no | |
123 | | -| <a name="input_slurm_operator_chart_version"></a> [slurm\_operator\_chart\_version](#input\_slurm\_operator\_chart\_version) | Version of the Slurm Operator chart to install. | `string` | `"0.2.1"` | no | |
| 159 | +| <a name="input_slurm_operator_chart_version"></a> [slurm\_operator\_chart\_version](#input\_slurm\_operator\_chart\_version) | Version of the Slurm Operator chart to install. | `string` | `"0.3.1"` | no | |
124 | 160 | | <a name="input_slurm_operator_namespace"></a> [slurm\_operator\_namespace](#input\_slurm\_operator\_namespace) | slurm namespace for charts | `string` | `"slinky"` | no | |
125 | 161 | | <a name="input_slurm_operator_repository"></a> [slurm\_operator\_repository](#input\_slurm\_operator\_repository) | Value overrides for the Slinky release | `string` | `"oci://ghcr.io/slinkyproject/charts"` | no | |
126 | 162 | | <a name="input_slurm_operator_values"></a> [slurm\_operator\_values](#input\_slurm\_operator\_values) | Value overrides for the Slinky release | `any` | `{}` | no | |
|
0 commit comments