|
1 | 1 | # Test Kubernetes Provider |
2 | 2 |
|
3 | | -`test-kubernetes` is a developer-only provider that backs cloud instance lifecycle calls with Kubernetes resources. |
| 3 | +`test-kubernetes` is a test/non-prod-only provider that backs cloud instance lifecycle calls with Kubernetes resources. |
4 | 4 |
|
5 | 5 | ## Credentials |
6 | 6 |
|
@@ -52,89 +52,4 @@ Clean up: |
52 | 52 |
|
53 | 53 | ```bash |
54 | 54 | minikube --profile testkube delete |
55 | | -``` |
56 | | - |
57 | | -### CI: kind |
58 | | - |
59 | | -kind remains the preferred CI path because it runs a disposable Kubernetes cluster on the GitHub Linux runner's Docker runtime. The CI path still uses `cloud-provider-kind` to provide `LoadBalancer` Services, but the provider itself only reads standard Kubernetes Service status. |
60 | | - |
61 | | -Minimal GitHub Actions sketch: |
62 | | - |
63 | | -```yaml |
64 | | -jobs: |
65 | | - testkube-validation: |
66 | | - runs-on: ubuntu-latest |
67 | | - steps: |
68 | | - - uses: actions/checkout@v4 |
69 | | - - uses: actions/setup-go@v5 |
70 | | - with: |
71 | | - go-version-file: go.mod |
72 | | - - name: Install cloud-provider-kind |
73 | | - run: go install sigs.k8s.io/cloud-provider-kind@latest |
74 | | - - name: Create kind cluster |
75 | | - run: | |
76 | | - kind create cluster --name testkube --wait 5m |
77 | | - kubectl label node testkube-control-plane node.kubernetes.io/exclude-from-external-load-balancers- || true |
78 | | - docker build -t ghcr.io/brevdev/cloud/testkube-ubuntu-vm:latest ./v1/providers/testkube/images/ubuntu-vm |
79 | | - kind load docker-image ghcr.io/brevdev/cloud/testkube-ubuntu-vm:latest --name testkube |
80 | | - kubectl create namespace testkube |
81 | | - echo "TESTKUBE_KUBECONFIG_BASE64=$(kind get kubeconfig --name testkube | base64 | tr -d '\n')" >> "$GITHUB_ENV" |
82 | | - echo "TESTKUBE_NAMESPACE=testkube" >> "$GITHUB_ENV" |
83 | | - echo "VALIDATION_TEST=1" >> "$GITHUB_ENV" |
84 | | - - name: Start cloud-provider-kind |
85 | | - run: | |
86 | | - sudo "$(go env GOPATH)/bin/cloud-provider-kind" > /tmp/cloud-provider-kind.log 2>&1 & |
87 | | - - name: Run provider validation |
88 | | - run: go test -v -run 'TestValidationFunctions|TestInstanceLifecycleValidation|TestFailureInstanceTypesValidation' ./v1/providers/testkube |
89 | | -``` |
90 | | -
|
91 | | -These tests validate inventory, Kubernetes resource creation/listing, SSH access, stop/start/delete, and configured failure types. They do not validate dev-plane software setup. |
92 | | -
|
93 | | -### Image-Backed SSH Validation |
94 | | -
|
95 | | -Build the local image: |
96 | | -
|
97 | | -```bash |
98 | | -docker build -t ghcr.io/brevdev/cloud/testkube-ubuntu-vm:latest ./v1/providers/testkube/images/ubuntu-vm |
99 | | -``` |
100 | | - |
101 | | -For local minikube validation, load the image into the minikube profile: |
102 | | - |
103 | | -```bash |
104 | | -minikube --profile testkube image load ghcr.io/brevdev/cloud/testkube-ubuntu-vm:latest |
105 | | -``` |
106 | | - |
107 | | -For CI kind validation, load the image into the kind cluster: |
108 | | - |
109 | | -```bash |
110 | | -kind load docker-image ghcr.io/brevdev/cloud/testkube-ubuntu-vm:latest --name testkube |
111 | | -``` |
112 | | - |
113 | | -Then run the focused SSH validation: |
114 | | - |
115 | | -```bash |
116 | | -go test -v -run TestImageBackedInstanceValidation ./v1/providers/testkube |
117 | | -``` |
118 | | - |
119 | | -`TestImageBackedInstanceValidation` creates a `test.ok.cpu` instance with the baked image tag, waits for the pod and load balancer to become ready, uses the provider-returned SSH endpoint, and verifies key-based SSH plus `sudo`, `apt-get`, and systemd basics. |
120 | | - |
121 | | -CI can add the same build/load step before running the image-backed test: |
122 | | - |
123 | | -```yaml |
124 | | - - name: Build local testkube image |
125 | | - run: docker build -t ghcr.io/brevdev/cloud/testkube-ubuntu-vm:latest ./v1/providers/testkube/images/ubuntu-vm |
126 | | - - name: Load local testkube image into kind |
127 | | - run: kind load docker-image ghcr.io/brevdev/cloud/testkube-ubuntu-vm:latest --name testkube |
128 | | - - name: Run image-backed validation |
129 | | - run: go test -v -run TestImageBackedInstanceValidation ./v1/providers/testkube |
130 | | -``` |
131 | | -
|
132 | | -## Image Contract |
133 | | -
|
134 | | -`test.ok.cpu` points at `ghcr.io/brevdev/cloud/testkube-ubuntu-vm:latest`. Before that image is published to GHCR, build the local image with that same tag. Once the image is published, local and CI validation can either pull it or keep building and loading the same tag for hermetic tests. |
135 | | - |
136 | | -The image at `images/ubuntu-vm` is expected to behave like a minimal Ubuntu VM for dev-plane: SSH access, `sudo`, `apt-get`, and systemd compatibility. |
137 | | - |
138 | | -The cloud provider does not lay down dev-plane software; dev-plane owns that. The image only needs to provide a base OS environment that dev-plane setup can use. |
139 | | - |
140 | | -Future exposure modes, such as fixed `NodePort` for one-off local debugging, should be separate instance type specs rather than credential fields. |
| 55 | +``` |
0 commit comments