Skip to content

Commit e30ba7e

Browse files
authored
add a model docs (#17)
Signed-off-by: Brian Duenas <brian.duenas@ibm.com>
1 parent e0ed7a3 commit e30ba7e

2 files changed

Lines changed: 543 additions & 45 deletions

File tree

README.md

Lines changed: 86 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
Finetune Controller is a robust and flexible system designed to manage and streamline the fine-tuning of machine learning models on Kubernetes, particularly within OpenShift clusters. This project leverages modern tools and workflows, enabling efficient development and deployment processes for AI-driven applications.
44

55
### Features
6+
67
- Local Development: Get started quickly with a streamlined setup process using uv, a high-performance Python package and project manager.
78
- OpenShift Integration: Simplify deployment and scaling with OpenShift-specific configurations and GPU support for intensive workloads.
89
- MongoDB Backend: Seamlessly connect to a local or cluster-based MongoDB database.
@@ -11,12 +12,15 @@ Finetune Controller is a robust and flexible system designed to manage and strea
1112
## Getting Started
1213

1314
If the cluster is already set up continue else follow the cluster setup instructions [here](#setup-openshift-cluster)
15+
1416
### Prereqs
15-
1. Recommend using [uv](https://github.com/astral-sh/uv), *an extremely fast Python package and project manager*
1617

17-
```shell
18-
pip install uv
19-
```
18+
1. Recommend using [uv](https://github.com/astral-sh/uv), _an extremely fast Python package and project manager_
19+
20+
```shell
21+
pip install uv
22+
```
23+
2024
2. A container engine such as Docker or Podman
2125

2226
<!-- ## Quick Setup using Compose
@@ -41,51 +45,59 @@ To stop the application:
4145
docker compose down
4246
``` -->
4347

44-
4548
### Install
49+
4650
1. Create virtual environment and install dependencies
47-
```shell
48-
uv sync
49-
```
5051

51-
2. Start a local developement mongo database *(or connect to one on cluster with port-forward)*
52+
```shell
53+
uv sync
54+
```
5255

53-
Local
54-
```shell
55-
docker run -d --rm --name mongodb \
56-
-e MONGODB_INITDB_ROOT_USERNAME="default-user" \
57-
-e MONGODB_INITDB_ROOT_PASSWORD="admin123456789" \
58-
-e MONGODB_INITDB_DATABASE="finetune" \
59-
-p 27017:27017 \
60-
mongodb/mongodb-community-server:latest
61-
```
56+
2. Start a local developement mongo database _(or connect to one on cluster with port-forward)_
6257

63-
you can port-forward this connection to your local machine
64-
```shell
65-
oc port-forward service/mongodb-community-server 27017:27017 -n <namespace>
66-
```
58+
Local
59+
60+
```shell
61+
docker run -d --rm --name mongodb \
62+
-e MONGODB_INITDB_ROOT_USERNAME="default-user" \
63+
-e MONGODB_INITDB_ROOT_PASSWORD="admin123456789" \
64+
-e MONGODB_INITDB_DATABASE="finetune" \
65+
-p 27017:27017 \
66+
mongodb/mongodb-community-server:latest
67+
```
68+
69+
you can port-forward this connection to your local machine
70+
71+
```shell
72+
oc port-forward service/mongodb-community-server 27017:27017 -n <namespace>
73+
```
6774

6875
3. Connect to the Openshift cluster with the cli login command `oc login`. If cluster not already set up follow [these](#setup-openshift-cluster) steps
6976

7077
4. Create a project level `.env` file (see `.env.example`) and update the variables.
71-
```shell
72-
cp .env.example .env
73-
```
78+
79+
```shell
80+
cp .env.example .env
81+
```
7482

7583
5. Make sure the virtual environment is activated and start the local finetuning controller application.
76-
```shell
77-
source .venv/bin/activate
7884

79-
uvicorn app.main:app --reload
80-
```
85+
```shell
86+
source .venv/bin/activate
87+
88+
uvicorn app.main:app --reload
89+
```
8190

8291
This will:
92+
8393
- Start MongoDB with the required configuration
8494
- Build and start the FastAPI server
8595
- Make the application available at http://localhost:8000
8696

8797
### Development and Contributing
98+
8899
Setup [pre-commit](https://pre-commit.com/#install) to keep linting and code styling up to standard.
100+
89101
```shell
90102
uv sync
91103
pre-commit install
@@ -94,57 +106,69 @@ pre-commit install
94106
## Setup OpenShift Cluster Resources
95107

96108
### Create default project
109+
97110
Name can be descriptive for these examples we will use `finetune-controller`
111+
98112
```shell
99113
oc new-project finetune-controller
100114
```
101115

102116
### Create Kubeflow project
117+
103118
```shell
104119
oc new-project kubeflow
105120
```
106121

107122
### Install Kubeflow training operator
123+
108124
<!-- ```shell
109125
kubectl apply -k "github.com/kubeflow/training-operator/manifests/overlays/standalone"
110126
``` -->
127+
111128
```shell
112129
kubectl apply --server-side -k "github.com/kubeflow/training-operator.git/manifests/overlays/standalone?ref=v1.8.1"
113130
```
114131

115132
### Install Kueue
133+
116134
> Requires Kubernetes 1.29 or newer
117135
118136
Follow the latest [docs](https://kueue.sigs.k8s.io/docs/installation/)
119137

120138
Install a released version
139+
121140
```shell
122141
kubectl apply --server-side -f https://github.com/kubernetes-sigs/kueue/releases/download/v0.10.1/manifests.yaml
123142
```
124143

125144
To wait for Kueue to be fully available, run:
145+
126146
```shell
127147
kubectl wait deploy/kueue-controller-manager -nkueue-system --for=condition=available --timeout=5m
128148
```
129149

130150
Restart pods
151+
131152
```shell
132153
kubectl delete pods -lcontrol-plane=controller-manager -nkueue-system
133154
```
134155

135156
First update the namepspace for the crd LocalQueue object in [default-user-queue.yaml](/crds/kueue/default-user-queue.yaml). default namepsace: "default"
157+
136158
```shell
137159
yq e '.metadata.namespace = "finetune-controller"' -i crds/kueue/default-user-queue.yaml
138160
```
139161

140162
Apply the default CRD config for Kueue or update by following their docs
163+
141164
```shell
142165
kubectl apply -f crds/kueue/
143166
```
144167

145168
### Install mongodb server
146169

147-
Example configuration. *do properly configure for production*
170+
Example configuration. _do properly configure for production_
171+
148172
```shell
149173
oc new-app -e MONGODB_INITDB_ROOT_USERNAME="default-user" -e MONGODB_INITDB_ROOT_PASSWORD="admin123456789" -e MONGODB_INITDB_DATABASE="finetune" mongodb/mongodb-community-server:latest --namespace finetune-controller
150174
```
@@ -165,16 +189,19 @@ oc set env deployment/ml-pipeline-ui DISABLE_GKE_METADATA=true
165189
``` -->
166190

167191
### Add GPU nodes to ROSA cluster
192+
168193
Go to your cluster on redhat console [admin dashboard](https://console.redhat.com/openshift/cluster-list). Add a machine pool of your choosing with the following configuration:
169194

170195
Taints
196+
171197
```
172198
key: nvidia.com/gpu
173199
value: <machine pool type or other>
174200
effect: NoSchedule
175201
```
176202

177203
Node Labels
204+
178205
```
179206
Key: cluster-api/accelerator
180207
Value: <gpu type e.g. V100 or empty>
@@ -183,6 +210,7 @@ Value: <gpu type e.g. V100 or empty>
183210
### Setup AWS Secret
184211

185212
Example aws config
213+
186214
```yaml
187215
# aws_credentials.yaml
188216
apiVersion: v1
@@ -198,13 +226,15 @@ type: Opaque
198226
```
199227
200228
Example for base 64 command in terminal
229+
201230
```bash
202231
echo -n "VALUE" | base64
203232
```
204233

205234
### Setup Pull secrets
206235

207236
Example docker pull secret config
237+
208238
```yaml
209239
# pull_secret.yaml
210240
apiVersion: v1
@@ -214,46 +244,57 @@ kind: Secret
214244
metadata:
215245
name: cr-pull-secret
216246
type: kubernetes.io/dockerconfigjson
217-
218247
```
219248
220249
Apply these secrets
250+
221251
```shell
222252
oc apply -f aws-credentials.yaml -n finetune-controller
223253
```
224254

225255
## Install Finetune Controller On OpenShift
226256

227257
1. Create a `.env.production` file and update the defaults. For this example set `MONGODB_URL=mongodb://mongodb-community-server.finetune-controller.svc.cluster.local:27017`
228-
```shell
229-
cp .env.example .env.production
230-
```
258+
259+
```shell
260+
cp .env.example .env.production
261+
```
231262

232263
2. create the application
233-
```shell
234-
oc new-app --strategy=docker --binary --name finetune-controller --env-file=".env.production" --namespace finetune-controller
235-
```
264+
265+
```shell
266+
oc new-app --strategy=docker --binary --name finetune-controller --env-file=".env.production" --namespace finetune-controller
267+
```
236268

237269
3. expose services and patch tls config
238-
```shell
239-
oc expose deployment/finetune-controller --port=8000
240-
oc expose svc/finetune-controller --port=8000
241-
oc patch route finetune-controller --type=merge -p '{"spec":{"tls":{"termination":"edge"}}}'
242-
```
270+
271+
```shell
272+
oc expose deployment/finetune-controller --port=8000
273+
oc expose svc/finetune-controller --port=8000
274+
oc patch route finetune-controller --type=merge -p '{"spec":{"tls":{"termination":"edge"}}}'
275+
```
243276

244277
4. add cluster role binding permissions to the application
245278

246279
5. start a build
247-
```shell
248-
oc start-build finetune-controller --from-dir=. --namespace=finetune-controller
249-
```
280+
```shell
281+
oc start-build finetune-controller --from-dir=. --namespace=finetune-controller
282+
```
250283

251284
## Manually Publish Updates To Finetune Controller
285+
252286
Publish From current project
287+
253288
```shell
254289
./scripts/publish.sh
255290
```
291+
256292
Publish From git ~HEAD
293+
257294
```shell
258295
./scripts/publish_git.sh
259296
```
297+
298+
# Adding Finetuning Models
299+
300+
look at [setting up a finetune model](/docs/setup_models.md)

0 commit comments

Comments
 (0)