Skip to content

Commit b682dc4

Browse files
byabazairejananos
authored andcommitted
docs: Add docs for the MLConnector
Signed-off-by: John Byabazaire <byabazairej@gmail.com>
1 parent e49b798 commit b682dc4

File tree

10 files changed

+930
-1
lines changed

10 files changed

+930
-1
lines changed

.DS_Store

4 KB
Binary file not shown.

docs/.nav.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,10 @@ nav:
66
- Installation: installation.md
77
- Design:
88
- design/*.md
9+
- MLConnector:
10+
- mlconnector/Overview.md
11+
- mlconnector/Installation.md
12+
- mlconnector/Step-by-step guide.md
913
- Developer Guide:
1014
- developer-guide/*.md
1115
- Tutorials:

docs/assets/img/MLConnector.png

692 KB
Loading

docs/mlconnector/Installation.md

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
## MLConnector Installation
2+
3+
This guide will walk you through setting up and running the MLConnector using Docker.
4+
5+
---
6+
7+
## Prerequisites
8+
9+
Before you begin, ensure you have the following installed on your system:
10+
11+
- **Docker**: Install Docker Engine and Docker Compose from [Docker’s official website](https://www.docker.com/).
12+
13+
---
14+
15+
## Environment Variables
16+
17+
The MLConnector relies on several external components. Define the following environment variables in your shell or an `.env` file:
18+
19+
### 1. Docker Registry
20+
The MLConnector dynamically creates and stores docker images for inference applications used within MYLSysOps. As such, it needs to to be able to communicate to a registry weather public, or private. This application was tested with docker registry. For further information on docker registry [check](https://docs.docker.com/get-started/docker-concepts/the-basics/what-is-a-registry/).
21+
22+
- `DOCKER_REGISTRY_ENDPOINT`: Your Docker registry endpoint
23+
- `DOCKER_USERNAME`: Your Docker registry username
24+
- `DOCKER_PASSWORD`: Your Docker registry password
25+
26+
### 2. AWS (File Storage)
27+
The MLConnector uses an external storage service, S3 to store it's data including training data and other files. You will need to setup and S3 bucket, or S3 compatible service to complete this setup. After, please provide the following details. If you do not have access to S3 bucket, or S3 compatible service, please contact us and we can help setup a temporarly one.
28+
- `AWS_ACCESS_URL`: AWS S3 endpoint URL
29+
- `AWS_ACCESS_KEY_ID`: AWS access key ID
30+
- `AWS_SECRET_ACCESS_KEY`: AWS secret access key
31+
- `AWS_S3_BUCKET_DATA`: Name of the S3 bucket for data
32+
33+
### 3. PostgreSQL Database
34+
This is used for internal communication of the varrious services. You can setup an external database service if you like. For simplicity you can you use the default values;
35+
- `POSTGRES_DB`: PostgreSQL database name (default, `mlmodel`)
36+
- `POSTGRES_USER`: PostgreSQL username (default, `postgres`)
37+
- `POSTGRES_PASSWORD`: PostgreSQL password (default, `strongpassword`)
38+
- `PGADMIN_DEFAULT_EMAIL`: pgAdmin default login email (default, `user@mail.com`)
39+
- `PGADMIN_DEFAULT_PASSWORD`: pgAdmin default login password (default, `strongpassword`)
40+
- `DB_HOST_NAME`: Database host (e.g., `database`, This corresponds to the name of the container)
41+
- `DB_PORT`: Database port (default: `5432`)
42+
- `DB_DRIVER`: Database driver string (default, `postgresql+asyncpg`) **NOTE:** Only use an async driver
43+
44+
### 4. Northbound API Endpoint
45+
The MLConnector communicates with part of the MYLSyops via the `NORTHBOUND_API`. Please set this value to the right endpoint.
46+
- `NORTHBOUND_API_ENDPOINT`: Base URL for the Northbound API (e.g., `http://your-host:8000`)
47+
48+
---
49+
50+
## Running the Application
51+
52+
1. **Start the Docker Containers**
53+
54+
```bash
55+
docker compose up -d
56+
```
57+
58+
This command builds and launches all required services in detached mode.
59+
60+
2. **View Container Logs**
61+
62+
```bash
63+
docker compose logs -f
64+
```
65+
66+
---
67+
68+
## Accessing the API Documentation
69+
70+
Once the services are up and running, open your browser and navigate to:
71+
72+
```
73+
http://<your-host>:8090/redoc
74+
```
75+
76+
Replace `<your-host>` with your server’s hostname or `localhost` if running locally.
77+
78+
---
79+
80+
## Troubleshooting
81+
82+
- **Port Conflicts**: Ensure ports `8090` (API docs) and your database port are available.
83+
- **Environment Variables**: Verify all required variables are set. Use `docker compose config` to inspect the interpolated configuration.
84+
- **Docker Connectivity**: Ensure Docker Engine is running and your user has permissions to run Docker commands.
85+
- **API Error Codes**: All status codes and error messages can be accessed via: `http://<your-host>:8090/redoc`
86+
87+
---
88+
89+
## License
90+
91+
***
92+
93+
---
94+

docs/mlconnector/Overview.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
## MLConnector
2+
This section describes the ML API (MLConnector) design. It is based on Flask REST. This is the bridge between all MYSysOps operations and ML assisted operations. It allow for flexible and decoupled way to train, deploy, and monitor all ML operations within the MYSysOps continuum. It also offers surpport for drift detectin and explainability. Below the flow diagram.
3+
4+
<div align="center">
5+
<img src="../img/MLConnector.png"
6+
alt="MLConnector Diagram"
7+
style="width:70%; height:auto;"/>
8+
</div>
9+
10+
For installation and step-by-step guide, please checkout the following sections.

0 commit comments

Comments
 (0)