Updated documentation and fixed a few things.

Fully updated all of the relevant documentation, as well as the main
README.md.

Fixed a bug in .gitignore where env configs were ignored, while some of
the log files were not.

Fixed a bug in the fake data generator that caused failure due to
poor datetime handling. This also implied a change in
ml/components/feature_engineering/arrival_date.py, where the feature
was not handled correctly, and assumed ideal scenarios.

Added some tests for the snapshot binding generator.

Updated pyproject.toml to ignore coverage for the fake data generator,
since it requires very specific packages, and is not a core part of the
repo anyway.

Added some new gifs and uml diagrams and updated the old ones, as a part
of the documentation update.

Included some artifacts and logs to be committed, so that the user can
immediately see the expected structure and content of some of the
relevant pipelines and scripts outputs, without having to run them first.
Only included a handful of them, to avoid cluttering the repo with too
many files.

Author: Sebastijan-Dominis

.gitignore

@@ -50,20 +50,20 @@ coverage.json
 *.py.cover
 .hypothesis/
 .pytest_cache/
-cover/
+/cover/
 
 # Translations
 *.mo
 *.pot
 
 # Django stuff:
-*.log
+# *.log
 local_settings.py
 db.sqlite3
 db.sqlite3-journal
 
 # Flask stuff:
-instance/
+/instance/
 .webassets-cache
 
 # Scrapy stuff:
@@ -74,7 +74,7 @@ docs/_build/
 
 # PyBuilder
 .pybuilder/
-target/
+/target/
 
 # Jupyter Notebook
 .ipynb_checkpoints
@@ -139,9 +139,9 @@ celerybeat.pid
 .env
 .envrc
 .venv
-env/
+/env/
 venv/
-ENV/
+/ENV/
 env.bak/
 venv.bak/
 
@@ -212,20 +212,22 @@ __marimo__/
 .history/
 
 # CatBoost model snapshot files
-catboost_info/
+/catboost_info/
 
 # VSCodeCounter cache
 .vscodecounter/
 
 # Failure management folder that helps deal with failed search and train runs by storing relevant information at runtime (best params from each search, training model snapshots) and deleting if the run is successful. Irrelevant for version control.
 /failure_management/
 
-# Ignore all experiments, data, feature sets, model registry & archive, predictions, and monitoring files. 
+# Ignore all experiments, data, feature sets, model registry & archive, predictions, and monitoring files, as well as log files. 
 # These are all potentially relevant for version control, but we want to temporarily ignore them while we work 
 # on the registry and experiment tracking features.
 /experiments/
 /data/
 /feature_store/
 /model_registry/
 /predictions/
-/monitoring/
+/monitoring/
+/orchestration_logs/
+/scripts_logs/

README.md

@@ -9,11 +9,12 @@
     - located in `data/raw/hotel_bookings/v1/2026-02-25T22-43-23_732dfdb7/data.csv`
     - originally from https://www.kaggle.com/datasets/mojtaba142/hotel-booking 
 - Current architecture expanded to support many datasets.
-- The ml workflow covers everything from the registration of a raw data snapshot to model promotion.
+- The ml workflow covers everything from the registration of a raw data snapshot to model monitoring.
 > Note: the repo was previously named `hotel_management`, so you will see that name around the repo; renamed for clarity on what the project does
 
 ## Features
 
+Pipelines for every part of the ml workflow:
 - Data preprocessing
   - Register raw data snapshots
   - Build interim and processed datasets
@@ -25,6 +26,25 @@
 - Model promotion
   - Includes model registry for staging and production
   - Archives past production models
+- Model inference
+- Model monitoring
+
+Maximum **decoupling** of datasets, feature sets, and modeling
+- Datasets merge at runtime, using predefined configs and DAG for ordering
+- Feature sets merge at runtime using a predefined entity key
+- Models can use any snapshots of datasets and feature sets via snapshot bindings registry
+- Validation ensures consistency and predefined minimum row presence
+
+Full **reproducibility**
+- Hashing and downstream validation of relevant `artifacts` and `configs`
+- Runtime info validation (hardware, git commit, environment...)
+
+Code **quality** ensured by CI, which includes:
+- `ruff` checks
+- `mypy` checks (moderate strictness)
+- import layer checks
+- naming conventions checks
+- **1235 tests** -> fails if coverage drops below 90%
 
 ## Installation
 
@@ -44,21 +64,33 @@ Two options:
 
 See the [usage guide](docs/usage.md) for instructions on running the workflow.
 
-### Usage examples (via `ml_service` and `Docker`):
+### Usage examples (via `ml_service`):
 
-#### Pipelines
+#### Modeling Configs Writing, Validation, Saving, and Viewing
 
-!["Gif portrayal of pipelines app from ml_service"](assets/gifs/ml_service_pipelines_v2.gif)
+!["Gif portrayal of writing modeling configs with ml_service"](assets/gifs/ml_service_modeling_configs_v3.gif)
 
-#### Modeling Configs
+**Similar functionality exists for other supported configs**
 
-!["Gif portrayal of modeling configs app from ml_service"](assets/gifs/ml_service_modeling_configs_v2.gif)
+#### Pipeline Running and Artifact Viewing
+
+!["Gif portrayal of running a pipeline with ml_service"](assets/gifs/ml_service_pipelines_v3.gif)
+
+**Similar functionality exists for scripts**
+
+#### Documentation Reading in Browser
+
+!["Gif portrayal of reading the docs with ml_service"](assets/gifs/ml_service_docs_v1.gif)
+
+#### Directory Structure Viewing in Browser
+
+["Gif portrayal of viewing directory structure with ml_service"](assets/gifs/ml_service_dir_viewer_v1.gif)
 
 ## Architecture
 
 ### Artifact Lineage (high-level overview)
 
-![Artifact Lineage Diagram](docs/architecture/img/artifact_lineage_v2.png)
+![Artifact Lineage Diagram](docs/architecture/img/artifact_lineage_v3.png)
 
 ### Details