Merge branch 'release/current' into main

Author: Kedae

.circleci/config.yml

@@ -133,3 +133,8 @@ dmypy.json
 
 # Cython debug symbols
 cython_debug/
+
+# custom
+# ignoring the poetry.lock files for now
+# as we don't have a good release scheme for keeping them up to date at the moment
+poetry.lock

.gitignore

@@ -10,6 +10,59 @@ The following repository is used to store the OpenAEV injectors for the platform
 
 This repository is used to host injectors that are supported by the core development team of OpenAEV. Nevertheless, the community is also developing a lot of injectors, third-parties modules directly linked to OpenAEV. You can find the list of all available injectors and plugins in the [OpenAEV ecosystem dedicated space](https://filigran.notion.site/OpenAEV-Ecosystem-30d8eb73d7d04611843e758ddef8941b).
 
+### Creating a new injector
+
+#### Project setup
+Assuming a new collector by the name of `new_injector`, create a skeleton directory with:
+```shell
+poetry new new_injector
+```
+
+#### `pyoaev` dependency
+We wish to retain the possibility to develop simultaneously on `pyoaev` and collectors. We rely on PEP 508 environment
+markers to alternatively install a local path `pyoaev` dependency or a released version from PyPI; specifically the `extra`
+marker.
+
+Navigate to the new directory and edit `pyproject.toml`.
+```shell
+vim new_injector/pyproject.toml
+```
+(or open the file in your favourite editor).
+
+Here's the expression for the pyoaev dependency, including the `extra` definition:
+```toml
+[tool.poetry.dependencies]
+pyoaev = [
+    { markers = "extra == 'prod' and extra != 'dev'", version = "<latest pyoaev release on PyPI>", source = "pypi"  },
+    { markers = "extra == 'dev' and extra != 'prod'", path = "../../client-python", develop = true },
+]
+
+[tool.poetry.extras]
+prod = ["pyoaev"]
+dev = ["pyoaev"]
+```
+
+### Simultaneous development on pyoaev and an injector
+The injectors repository is set to assume that in the event of a simultaneous development work on both `pyoaev`
+and injectors, the `pyoaev` repository is cloned in a directory at the same level as the injectors root directory,
+and is named strictly `client-python`.
+
+Here's an example layout:
+```
+.
+├── client-python       <= mandatory dir name
+│   ├── docs
+│   ├── pyoaev
+│   ├── scripts
+│   └── test
+└── injectors          <= this repo root dir
+    ├── aws
+    ├── http-query
+    ├── nmap
+    └── nuclei
+```
+
+
 ## Contributing
 
 If you want to help use improve or develop new injector, please check out the **[development documentation for new injectors](https://docs.openaev.io/latest/development/injectors)**. If you want to make your injectors available to the community, **please create a Pull Request on this repository**, then we will integrate it to the CI and in the [OpenAEV ecosystem](https://filigran.notion.site/OpenAEV-Ecosystem-30d8eb73d7d04611843e758ddef8941b).

README.md

@@ -1,35 +1,46 @@
-FROM python:3.11-slim
+FROM python:3.13-alpine AS builder
 
-# Install system dependencies
-RUN apt-get update && apt-get install -y \
-    git \
-    gcc \
-    python3-dev \
-    libssl-dev \
-    libffi-dev \
-    && rm -rf /var/lib/apt/lists/*
+RUN apk update && apk upgrade
 
-# Create working directory
-WORKDIR /opt/openaev-injector-aws
+WORKDIR /opt/injector_common
+COPY --from=injector_common ./ ./
 
-# Copy the injector source code
-COPY src /opt/openaev-injector-aws
+# poetry version available on Ubuntu 24.04
+RUN pip3 install poetry==2.1.3
 
-# Install Python dependencies
-RUN pip3 install --no-cache-dir -r requirements.txt
+ARG installdir=/opt/injector
+ADD . ${installdir}
+WORKDIR ${installdir}
+RUN poetry build
+
+FROM python:3.13-alpine AS runner
+
+WORKDIR /opt/injector_common
+COPY --from=injector_common ./ ./
+
+ARG installdir=/opt/injector
+WORKDIR ${installdir}
+COPY --from=builder ${installdir} ${installdir}
+RUN pip3 install --no-cache-dir "$(ls dist/*.whl)[prod]"
+
+# Declare the build argument
+ARG PYOAEV_GIT_BRANCH_OVERRIDE
+
+RUN if [[ ${PYOAEV_GIT_BRANCH_OVERRIDE} ]] ; then \
+        echo "Forcing specific version of client-python" && \
+        apk add --no-cache git && \
+        pip install pip3-autoremove && \
+        pip-autoremove pyoaev -y && \
+        pip install git+https://github.com/OpenAEV-Platform/client-python@${PYOAEV_GIT_BRANCH_OVERRIDE} ; \
+    fi
 
 # Verify AWS CLI is installed
 RUN aws --version || echo "AWS CLI installation verification"
 
-# Create AWS data directory
-RUN mkdir -p /root/.local/share/aws
-
 # Set environment variables for AWS
 ENV AWS_HOME=/root/.local/share/aws
+# Create AWS data directory
+RUN mkdir -p ${AWS_HOME}
 
-# Copy and set up entrypoint script
-COPY entrypoint.sh /
-RUN chmod +x /entrypoint.sh
 
-# Set the entrypoint
-ENTRYPOINT ["/entrypoint.sh"]
+CMD ["python3", "-m", "aws.openaev_aws"]

aws/Dockerfile

@@ -85,14 +85,39 @@ The injector supports all current AWS regions including:
 
 **Other Regions**: Middle East (Bahrain, UAE, Tel Aviv), Africa (Cape Town), South America (São Paulo), China (Beijing, Ningxia), AWS GovCloud (US-East, US-West)
 
-## Installation
+## Configuration variables
+
+There are a number of configuration options, which are set either in `docker-compose.yml` (for Docker) or
+in `config.yml` (for manual deployment).
+
+### OpenAEV environment variables
+
+Below are the parameters you'll need to set for OpenAEV:
+
+| Parameter     | config.yml | Docker environment variable | Mandatory | Description                                          |
+|---------------|------------|-----------------------------|-----------|------------------------------------------------------|
+| OpenAEV URL   | url        | `OPENAEV_URL`               | Yes       | The URL of the OpenAEV platform.                     |
+| OpenAEV Token | token      | `OPENAEV_TOKEN`             | Yes       | The default admin token set in the OpenAEV platform. |
+
+### Base injector environment variables
+
+Below are the parameters you'll need to set for running the injector properly:
+
+| Parameter        | config.yml | Docker environment variable | Default | Mandatory | Description                                                                            |
+|------------------|------------|-----------------------------|---------|-----------|----------------------------------------------------------------------------------------|
+| Injector ID      | id         | `INJECTOR_ID`               | /       | Yes       | A unique `UUIDv4` identifier for this injector instance.                               |
+| Collector Name   | name       | `INJECTOR_NAME`             |         | Yes       | Name of the injector.                                                                  |
+| Log Level        | log_level  | `INJECTOR_LOG_LEVEL`        | info    | Yes       | Determines the verbosity of the logs. Options are `debug`, `info`, `warn`, or `error`. |
+
+
+## Deployment
 
 ### Using Docker
 
 1. Build the Docker image:
 ```bash
 cd aws
-docker build -t openaev/injector-aws:latest .
+docker build --build-context injector_common=../injector_common -t openaev/injector-aws:latest .
 ```
 
 2. Run with docker-compose:
@@ -102,54 +127,35 @@ docker-compose up -d
 
 ### Manual Installation
 
-1. Install Python dependencies:
-```bash
-cd aws/src
-pip install -r requirements.txt
-```
-
-2. Install AWS:
-```bash
-pip install aws
-```
-
-3. Configure the injector:
-```bash
-cp config.yml.sample config.yml
-# Edit config.yml with your OpenAEV connection details
-```
-
-4. Run the injector:
-```bash
-python openaev_aws.py
-```
+Create a file `config.yml` based on the provided `config.yml.sample`.
 
-## Configuration
+Replace the configuration variables with the appropriate configurations for
+you environment.
 
-### Environment Variables
+The poetry package management system (version 2.1 or later) must also be available: https://python-poetry.org/
 
-- `OPENAEV_URL`: URL of your OpenAEV instance
-- `OPENAEV_TOKEN`: Authentication token for OpenAEV API
-- `INJECTOR_ID`: Unique identifier for this injector instance
-- `INJECTOR_NAME`: Display name for the injector (default: "AWS")
-- `INJECTOR_LOG_LEVEL`: Logging level (info, warning, error) - debug logging has been removed for production use
+Install the environment:
 
-### Configuration File
+**Production**:
+```shell
+# production environment
+poetry install --extras prod
+```
 
-Create a `config.yml` file based on the provided sample:
+**Development** (note that you should also clone the [pyoaev](OpenAEV-Platform/client-python) repository [according to
+these instructions](../README.md#simultaneous-development-on-pyoaev-and-an-injector))
+```shell
+# development environment
+poetry install --extras dev
+```
 
-```yaml
-openaev:
-  url: 'http://localhost:3001'
-  token: 'your-openaev-token'
+Then, start the collector:
 
-injector:
-  id: 'unique-injector-id'
-  name: 'AWS'
-  log_level: 'info'
+```shell
+poetry run python -m aws.openaev_aws
 ```
 
-## Usage in OpenAEV
+## Behaviour
 
 1. **Deploy the Injector**: Start the AWS injector using Docker or manual installation
 2. **Verify Registration**: Check that the injector appears in OpenAEV under Integrations > Injectors
@@ -221,7 +227,7 @@ To add support for additional AWS modules:
 
 Run the injector with info logging to test new modules:
 
-```python
+```yaml
 # In config.yml
 injector:
   log_level: 'info'

aws/README.md

@@ -0,0 +1,27 @@
+from pydantic import Field
+from pyoaev.configuration import ConfigLoaderOAEV, Configuration, SettingsLoader
+
+from aws.configuration.injector_config_override import InjectorConfigOverride
+from aws.contracts_aws import AWSContracts
+
+
+class ConfigLoader(SettingsLoader):
+    openaev: ConfigLoaderOAEV = Field(default_factory=ConfigLoaderOAEV)
+    injector: InjectorConfigOverride = Field(default_factory=InjectorConfigOverride)
+
+    def to_daemon_config(self) -> Configuration:
+        return Configuration(
+            config_hints={
+                # OpenAEV configuration (flattened)
+                "openaev_url": {"data": str(self.openaev.url)},
+                "openaev_token": {"data": self.openaev.token},
+                # Injector configuration (flattened)
+                "injector_id": {"data": self.injector.id},
+                "injector_name": {"data": self.injector.name},
+                "injector_type": {"data": "openaev_aws"},
+                "injector_contracts": {"data": AWSContracts.build_contract()},
+                "injector_log_level": {"data": self.injector.log_level},
+                "injector_icon_filepath": {"data": self.injector.icon_filepath},
+            },
+            config_base_model=self,
+        )

aws/src/__init__.py aws/aws/__init__.pyaws/src/__init__.py renamed to aws/aws/__init__.py

@@ -0,0 +1,17 @@
+from pydantic import Field
+from pyoaev.configuration import ConfigLoaderCollector
+
+
+# To be change ConfigLoaderCollector
+class InjectorConfigOverride(ConfigLoaderCollector):
+    id: str = Field(
+        description="A unique UUIDv4 identifier for this injector instance.",
+    )
+    name: str = Field(
+        default="AWS",
+        description="Name of the injector.",
+    )
+    icon_filepath: str | None = Field(
+        default="aws/img/icon-aws.png",
+        description="Path to the icon file",
+    )