|
1 | | -# m4.2_advanced_middleware_api |
| 1 | +# FAIRagro SQL-to-ARC Middleware |
2 | 2 |
|
3 | | -The API component of the advanced middleware that accepts ARCs in RO-Create format and pushes them to the datahub |
| 3 | +This repository contains the **SQL-to-ARC Converter**, a core component of the FAIRagro advanced middleware architecture. It enables Research Data Infrastructure (RDI) providers to transform their relational metadata into standardized Annotated Research Context (ARC) objects and transmit them to the central FAIRagro Middleware API. |
| 4 | + |
| 5 | +## 📁 Repository Structure |
| 6 | + |
| 7 | +| Folder | Description | |
| 8 | +| :--- | :--- | |
| 9 | +| [middleware/](middleware/) | Source code of the converter component. | |
| 10 | +| [docs/](docs/) | Architectural design, database view specifications, and API documentation. | |
| 11 | +| [dev_environment/](dev_environment/) | Docker-based local development setup (Postgres, Mock API). | |
| 12 | +| [scripts/](scripts/) | Tooling for quality checks, environment setup, and Git LFS. | |
| 13 | +| [docker/](docker/) | Dockerfiles and container structure tests. | |
| 14 | + |
| 15 | +## 🚀 Getting Started (Development) |
| 16 | + |
| 17 | +This project uses [uv](https://github.com/astral-sh/uv) for dependency management and workspace orchestration. |
| 18 | + |
| 19 | +### 1. Prerequisites |
| 20 | + |
| 21 | +- **Python 3.12+** |
| 22 | +- **Docker & Docker Compose** |
| 23 | +- **Git LFS** (installed via `./scripts/setup-git-lfs.sh`) |
| 24 | + |
| 25 | +### 2. Environment Setup |
| 26 | + |
| 27 | +Clone the repository and install all workspace dependencies: |
| 28 | + |
| 29 | +```bash |
| 30 | +uv sync --all-packages |
| 31 | +``` |
| 32 | + |
| 33 | +### 3. Start Local Development Environment |
| 34 | + |
| 35 | +The `dev_environment` folder provides a full stack including a PostgreSQL database pre-filled with edaphobase data: |
| 36 | + |
| 37 | +```bash |
| 38 | +cd dev_environment |
| 39 | +./start.sh --build |
| 40 | +``` |
| 41 | + |
| 42 | +This will start the database and run a test iteration of the converter. |
| 43 | + |
| 44 | +## 🔧 Component Documentation |
| 45 | + |
| 46 | +Detailed information on how to use, configure, and deploy the specific components can be found in their respective subdirectories: |
| 47 | + |
| 48 | +- **[SQL-to-ARC Converter README](middleware/sql_to_arc/README.md)**: Configuration (YAML/Env), CLI options, and production deployment. |
| 49 | +- **[Architectural Design](docs/ARCHITECTURAL_DESIGN.md)**: Deep dive into the concurrency model, memory management, and data flow. |
| 50 | +- **[Database View Spec](docs/sql_to_arc_database_views.md)**: The SQL views required for the RDI provider database. |
| 51 | + |
| 52 | +## 🧪 Quality Standards |
| 53 | + |
| 54 | +We maintain high code quality through automated checks: |
| 55 | + |
| 56 | +```bash |
| 57 | +# Run all quality checks (Ruff, Mypy, Pylint, Bandit) |
| 58 | +./scripts/quality-check.sh |
| 59 | + |
| 60 | +# Run unit and integration tests |
| 61 | +uv run pytest middleware/sql_to_arc/tests/ |
| 62 | +``` |
| 63 | + |
| 64 | +--- |
| 65 | +**Maintained by:** FAIRagro Middleware Team |
| 66 | +**License:** [LICENSE](LICENSE) |
0 commit comments