Skip to content

Commit 5a12a97

Browse files
Updated docs, improved UI, fixed minor bugs.
Updated the documentation to provide information on scripts and snapshot bindings registry. Updated setup and usage instructions. Altered a few other parts of the docs for clarity. Improved the GUI by adding descriptions to each page, highlighting optional vs required fields, and adding some styling to make it more visually appealing. Updated the Dockerfile and requirements.txt to not include sdv and a specific torch version required by generate_fake_data.py when training the CTGAN model on RTX 5070 Ti GPU, but still included the two packages as commented out lines, along with some comments for clarity. Fixed minor bugs in the code related to the above changes. Note: the gifs have not been updated yet, as more changes are expected soon, but they will be updated in the near future to reflect the new UI.
1 parent 9a4b58d commit 5a12a97

34 files changed

Lines changed: 1136 additions & 79 deletions

Dockerfile

Lines changed: 17 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
# ===== Base image with GPU support =====
2-
# FROM pytorch/pytorch:2.10.0-cuda12.8-cudnn9-runtime
3-
FROM nvidia/cuda:12.8.0-runtime-ubuntu22.04
2+
FROM pytorch/pytorch:2.10.0-cuda12.8-cudnn9-runtime
3+
# NOTE: Uncomment the line below and comment out the line above only if you want to generate fake data. For regular use, stick to the current image. If you use the image below, make sure to also uncomment the torch installation line in this Dockerfile. Likewise, uncomment the sdv dependency in requirements.txt.
4+
# FROM nvidia/cuda:12.8.0-runtime-ubuntu22.04
45

56
# ===== Set working directory =====
67
WORKDIR /app
@@ -23,28 +24,29 @@ ENV PATH="/opt/conda/bin:$PATH"
2324
RUN conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/main && \
2425
conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/r
2526

26-
# ===== Copy dependency files first for faster rebuilds =====
27+
# ===== Copy environment.yml first for faster rebuilds =====
2728
COPY environment.yml /tmp/environment.yml
28-
COPY requirements.txt /tmp/requirements.txt
29-
COPY setup.py .
30-
COPY pyproject.toml .
3129

3230
# ===== Create Conda environment =====
3331
# Create env
3432
RUN conda env create -f /tmp/environment.yml -n hotel_management
3533

34+
# Copy the rest of the files needed for installation
35+
COPY requirements.txt /tmp/requirements.txt
36+
COPY setup.py .
37+
COPY pyproject.toml .
38+
3639
# Install torch (nightly CUDA 12.8)
3740
# This version works with Nvidia RTX 5070 Ti GPU. If you experience issues, change to a compatible version for your GPU.
38-
RUN conda run -n hotel_management pip install --pre \
39-
torch==2.12.0.dev20260320+cu128 \
40-
--index-url https://download.pytorch.org/whl/nightly/cu128
41+
42+
# NOTE: Uncomment this only if you want to generate fake data. In that case, use the commented out base image at the top of this file. Likewise, uncomment the sdv dependency in requirements.txt. For regular use, stick to the current image.
43+
# RUN conda run -n hotel_management pip install --pre \
44+
# torch==2.12.0.dev20260320+cu128 \
45+
# --index-url https://download.pytorch.org/whl/nightly/cu128
4146

4247
# Install rest
4348
RUN conda run -n hotel_management pip install -r /tmp/requirements.txt
4449

45-
# Install your package
46-
RUN conda run -n hotel_management pip install -e .
47-
4850
# ===== Use the environment for all container commands =====
4951
SHELL ["conda", "run", "-n", "hotel_management", "/bin/bash", "-c"]
5052

@@ -54,6 +56,9 @@ COPY pipelines ./pipelines
5456
COPY scripts ./scripts
5557
COPY ml_service ./ml_service
5658

59+
# Install the package
60+
RUN conda run -n hotel_management pip install -e .
61+
5762
# ===== Expose ports =====
5863
EXPOSE 8000
5964
EXPOSE 8050

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ Two options:
3838
- operate the workflow through a `browser`
3939
- **Manual use**
4040
- requires `python` and `conda` (preferably)
41-
- operate the workflow through a `browser` or `cli + manual` (configs)
41+
- operate the workflow through a `browser` or `cli + manual (configs)`
4242

4343
## Usage
4444

docs/configs/bindings.md

Lines changed: 171 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,171 @@
1+
# Snapshot Bindings Registry
2+
3+
Snapshot bindings registry defines which snapshot of each specified dataset
4+
and feature set will be used by a script that uses these configs.
5+
6+
It allows the use of specific snapshots, hence enabling reproducibility.
7+
8+
Unlike some of the other configuration types, snapshot bindings are **centralized in a single configuration file** and selected dynamically at runtime of a given pipeline (likely `freeze.py`,
9+
`search.py`, or `train.py`).
10+
11+
## 1. Location
12+
13+
Snapshot bindings are defined in a single file:
14+
15+
```text
16+
configs/snapshot_bindings_registry/bindings.yaml
17+
```
18+
19+
This file contains all of the defined **snapshots of snapshot bindings**.
20+
21+
## 2. Configuration Structure
22+
23+
The configuration is organized hierarchically
24+
25+
General structure:
26+
27+
```text
28+
snapshot_id
29+
└─ snapshot_bindings
30+
```
31+
32+
Example:
33+
34+
```yaml
35+
2026-03-20T02-54-47_61509023:
36+
datasets:
37+
hotel_bookings:
38+
v1:
39+
snapshot: 2026-03-17T03-26-25_4e817323
40+
v2:
41+
snapshot: 2026-03-17T03-26-27_c04ca84b
42+
feature_sets:
43+
booking_context_features:
44+
v1:
45+
snapshot: 2026-03-17T06-41-04_530962ff
46+
v2:
47+
snapshot: 2026-03-17T04-27-05_b9099899
48+
v3:
49+
snapshot: 2026-03-17T04-27-07_6ecc2061
50+
v4:
51+
snapshot: 2026-03-17T04-27-08_e45a7462
52+
v5:
53+
snapshot: 2026-03-17T04-27-09_72290e7c
54+
v6:
55+
snapshot: 2026-03-17T04-27-10_f299b9d2
56+
v7:
57+
snapshot: 2026-03-17T04-27-12_eba7203e
58+
channel_and_agent_features:
59+
v1:
60+
snapshot: 2026-03-17T04-27-24_d2b1a0df
61+
v2:
62+
snapshot: 2026-03-17T04-27-25_1bd99237
63+
v3:
64+
snapshot: 2026-03-17T04-27-27_e73d2e8c
65+
customer_history_features:
66+
v1:
67+
snapshot: 2026-03-17T04-27-13_47665433
68+
v2:
69+
snapshot: 2026-03-17T04-27-14_1881a909
70+
v3:
71+
snapshot: 2026-03-17T04-27-15_c7be8caf
72+
v4:
73+
snapshot: 2026-03-17T04-27-17_1c0339c3
74+
pricing_party_features:
75+
v1:
76+
snapshot: 2026-03-17T04-27-18_2e72ae25
77+
v2:
78+
snapshot: 2026-03-17T04-27-19_dd815223
79+
v3:
80+
snapshot: 2026-03-17T04-27-20_0592cbb0
81+
v4:
82+
snapshot: 2026-03-17T04-27-21_c2f019fb
83+
v5:
84+
snapshot: 2026-03-17T04-27-23_8ef8fcc4
85+
room_allocation_features:
86+
v1:
87+
snapshot: 2026-03-17T04-27-28_0368fd56
88+
v2:
89+
snapshot: 2026-03-17T04-27-29_e07095bc
90+
```
91+
92+
Required snapshots are selected based on snapshot id at runtime. Extra dataset and feature set snapshots do not break the execution. Missing ones do.
93+
94+
## 3. Fields
95+
96+
### Dataset Snapshot Bindings
97+
98+
Dataset snapshot bindings follow the following nesting policy:
99+
100+
```text
101+
datasets:
102+
└─ {dataset_name}
103+
└─ {dataset_version}
104+
└─ snapshot_information
105+
```
106+
107+
Each nested layer specifies which snapshot should be used.
108+
109+
Example:
110+
111+
```yaml
112+
datasets:
113+
hotel_bookings:
114+
v1:
115+
snapshot: 2026-03-17T03-26-25_4e817323
116+
v2:
117+
snapshot: 2026-03-17T03-26-27_c04ca84b
118+
```
119+
120+
### Feature Set Snapshot Bindings
121+
122+
Feature set snapshot bindings follow the following nesting policy:
123+
124+
```text
125+
feature_sets:
126+
└─ {feature_set_name}
127+
└─ {feature_set_version}
128+
└─ snapshot_information
129+
```
130+
131+
Each nested layer specifies which snapshot should be used.
132+
133+
Example:
134+
135+
```yaml
136+
feature_sets:
137+
booking_context_features:
138+
v1:
139+
snapshot: 2026-03-17T06-41-04_530962ff
140+
v2:
141+
snapshot: 2026-03-17T04-27-05_b9099899
142+
v3:
143+
snapshot: 2026-03-17T04-27-07_6ecc2061
144+
v4:
145+
snapshot: 2026-03-17T04-27-08_e45a7462
146+
v5:
147+
snapshot: 2026-03-17T04-27-09_72290e7c
148+
v6:
149+
snapshot: 2026-03-17T04-27-10_f299b9d2
150+
v7:
151+
snapshot: 2026-03-17T04-27-12_eba7203e
152+
channel_and_agent_features:
153+
v1:
154+
snapshot: 2026-03-17T04-27-24_d2b1a0df
155+
v2:
156+
snapshot: 2026-03-17T04-27-25_1bd99237
157+
v3:
158+
snapshot: 2026-03-17T04-27-27_e73d2e8c
159+
```
160+
161+
## 4. Validation
162+
163+
Snapshot bindings are validated using the `validate_snapshot_binding_registry` and
164+
`validate_snapshot_binding` functions.
165+
166+
Validation ensures:
167+
- the config file is formatted properly
168+
- snapshot is not empty
169+
- snapshot contains dataset or feature set bindings, if required
170+
171+
Invalid configurations raise a `ConfigError`.

docs/configs/feature_registry.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -150,6 +150,8 @@ Validation ensures that every column listed in columns appears exactly once acro
150150

151151
Defines feature engineering operators applied during feature freezing.
152152

153+
**Operator hash** is generated via a script found in `scripts/generators/generate_operator_hash.py`.
154+
153155
Operators can either:
154156
- materialize features into the dataset
155157
- define logical feature relationships

docs/configs/promotion.md

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Promotion Thresholds
22

3-
Promotion thresholds defines the **minimum performance thresholds** a trained model must meet in order to be promoted (for example from candidate → production).
3+
Promotion thresholds registry defines the **minimum performance thresholds** a trained model must meet in order to be promoted (for example from candidate → production).
44

55
Promotion rules act as a **safety gate** to prevent models with insufficient performance from being promoted.
66

@@ -27,15 +27,13 @@ General structure:
2727
```text
2828
problem_type
2929
└─ segment
30-
└─ promotion policy
30+
└─ promotion_policy
3131
```
3232

3333
Example:
3434

3535
```yaml
36-
classification:
37-
38-
cancellation:
36+
cancellation:
3937

4038
global:
4139

docs/setup.md

Lines changed: 19 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -65,23 +65,37 @@ In `{repo_root}/.env`.
6565
### Environment Variables
6666

6767
`ML_SERVICE_BACKEND_URL` = URL in which you want to run the `FastAPI` backend from `ml_service`
68-
`ML_SERVICE_FRONTEND_URLS` = URLS you want to allow access in the backend
68+
`ML_SERVICE_FRONTEND_URL` = URL in which you want to run the `Dash` frontend from `ml_service`
6969

7070
Notes:
7171
- keep default frontend urls unless you change them in the frontend as well, or want to add more
72-
- frontend ports currently explicitly defined in the frontend code -> change if security becomes an issue
72+
- frontend port currently explicitly defined in the frontend code -> change if security becomes a concern
7373

7474
### Defaults
7575

76-
`ML_SERVICE_BACKEND_URL`= http://localhost:8000
77-
`ML_SERVICE_FRONTEND_URLS`= [http://localhost:8050, http://localhost:8051, http://localhost:8052, http://localhost:8053, http://localhost:8054, http://localhost:8055]
76+
`ML_SERVICE_BACKEND_URL`=http://localhost:8000
77+
`ML_SERVICE_FRONTEND_URL`=http://localhost:8050
78+
79+
## Fake data generation
80+
81+
Fake data can be generated using a script found in `scripts/generators/generate_fake_data.py`.
82+
If you want to do so, make sure to either:
83+
1. uncomment and install the two commented-out packages in `requirements.txt`
84+
2. uncomment `sdv` in `requirements.txt` and the relevant parts of the `Dockerfile` (if using `Docker`)
85+
86+
> Note: the code in that script is not modularized, as this repo does not focus on fake data generation,
87+
> and instead uses fake data generation only as a utility for quick and easy simulation and testing.
88+
89+
- The repo should include some fake data at any given point, along with a trained fake data generation model.
90+
- Only use this feature if necessary - the packages (sdv + torch) can act oddly depending on hardware, and
91+
tend to take a while to install.
7892

7993
## Post-installation
8094

8195
You can now operate the ml workflow in following ways
8296
- using `ml_service` in browser
8397
- backend on localhost:8000 (default)
84-
- frontend on localhost:{8050-8055} (split for easier maintenance; `Dash` scales poorly)
98+
- frontend on localhost:8050
8599
- manually
86100
- cli for pipelines
87101
- manual writing of configs

0 commit comments

Comments
 (0)