feat(flowtriq): add external-import connector for DDoS incidents#6822
Open
jacob-masse wants to merge 4 commits into
Open
feat(flowtriq): add external-import connector for DDoS incidents#6822jacob-masse wants to merge 4 commits into
jacob-masse wants to merge 4 commits into
Conversation
Contributor License Agreement✅ CLA signed 💚 Thank you @jacob-masse for signing the Contributor License Agreement! Your pull request can now be reviewed and merged. We appreciate your contribution to Filigran's open source projects! ❤️ This is an automated message from the Filigran CLA Bot. |
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new external-import connector (external-import/flowtriq/) that ingests Flowtriq DDoS incident data via the REST API and publishes STIX 2.1 bundles to OpenCTI, following the repository’s standard external-import connector structure and metadata conventions.
Changes:
- Introduces Flowtriq connector settings (Pydantic v2 via
connectors-sdk), API client, connector runtime loop, and STIX conversion logic for target/source IPs and indicators. - Adds containerization and deployment assets (Dockerfile, docker-compose, env/config samples).
- Adds connector metadata (manifest, config schema, config doc) for platform integration.
Reviewed changes
Copilot reviewed 17 out of 18 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| external-import/flowtriq/src/requirements.txt | Adds Python dependencies for the new connector runtime. |
| external-import/flowtriq/src/main.py | Provides the connector entrypoint to load settings and run the connector. |
| external-import/flowtriq/src/external_import_connector/settings.py | Defines connector + Flowtriq-specific configuration models. |
| external-import/flowtriq/src/external_import_connector/converter_to_stix.py | Converts Flowtriq incidents into STIX observables/indicators/relationships. |
| external-import/flowtriq/src/external_import_connector/connector.py | Implements scheduling, collection, filtering, and bundle submission to OpenCTI. |
| external-import/flowtriq/src/external_import_connector/client_api.py | Implements Flowtriq REST API client and pagination. |
| external-import/flowtriq/src/external_import_connector/init.py | Exposes connector classes for import usage. |
| external-import/flowtriq/src/config.yml.sample | Adds sample YAML configuration for local/manual runs. |
| external-import/flowtriq/README.md | Documents purpose, configuration, and deployment. |
| external-import/flowtriq/entrypoint.sh | Provides a shell entrypoint script for running the connector. |
| external-import/flowtriq/Dockerfile | Builds an Alpine-based Python 3.12 image for the connector. |
| external-import/flowtriq/docker-compose.yml | Provides a docker-compose service definition for the connector. |
| external-import/flowtriq/.env.sample | Provides a sample environment variable configuration. |
| external-import/flowtriq/.dockerignore | Excludes local config/cache/log artifacts from Docker build context. |
| external-import/flowtriq/metadata/connector_manifest.json | Registers connector metadata (name/slug/versions/image/etc.). |
| external-import/flowtriq/metadata/connector_config_schema.json | Provides generated config schema for environment variables. |
| external-import/flowtriq/metadata/CONNECTOR_CONFIG_DOC.md | Provides generated documentation for configuration parameters. |
| top_sources = incident.get("sp_top_sources") | ||
| if top_sources and isinstance(top_sources, list): | ||
| for source in top_sources: | ||
| source_ip = source.get("ip") if isinstance(source, dict) else str(source) |
Comment on lines
+133
to
+138
| objects = self.converter_to_stix.create_incident_observable(incident) | ||
| if objects: | ||
| stix_objects.extend(objects) | ||
| processed += 1 | ||
|
|
||
| self.helper.connector_logger.info( |
| } | ||
| ) | ||
|
|
||
| def get_incidents(self, limit: int = 50, offset: int = 0) -> dict | None: |
Comment on lines
+46
to
+50
| if self.config.flowtriq.incident_severity: | ||
| # API accepts a single severity filter; if multiple are configured, | ||
| # we make separate calls per severity in the connector layer. | ||
| # Here we pass the first one if set. | ||
| params["severity"] = self.config.flowtriq.incident_severity[0] |
Comment on lines
+94
to
+114
| all_incidents: list[dict] = [] | ||
| offset = 0 | ||
| page_size = min(max_total, 100) | ||
|
|
||
| while offset < max_total: | ||
| data = self.get_incidents(limit=page_size, offset=offset) | ||
| if not data or "incidents" not in data: | ||
| break | ||
|
|
||
| incidents = data["incidents"] | ||
| if not incidents: | ||
| break | ||
|
|
||
| all_incidents.extend(incidents) | ||
| offset += len(incidents) | ||
|
|
||
| # Stop if we got fewer than requested (last page) | ||
| if len(incidents) < page_size: | ||
| break | ||
|
|
||
| return all_incidents[:max_total] |
added 2 commits
June 29, 2026 08:55
Adds a new external-import connector that fetches DDoS attack incident data from the Flowtriq REST API and converts it to STIX 2.1 threat intelligence for ingestion into OpenCTI. The connector polls GET /api/v1/incidents on a configurable schedule, creates IPv4/IPv6 observables and indicators for attack targets and sources, and supports filtering by severity, status, and TLP marking. Follows the abuseipdb-ipblacklist connector pattern: Pydantic v2 settings, connectors-sdk base classes, and the standard project layout.
- Remove unused `confidence` variable (flake8 F841) - Break long line in source IP extraction for Black compliance - Persist `last_incident_time` in connector state so deduplication works across runs - Add `severity` parameter to `get_incidents()` so callers can request a specific severity value - Iterate all configured severities in `get_all_incidents()` instead of silently dropping all but the first
jacob-masse
force-pushed
the
feat/flowtriq-external-import
branch
from
b857809 to
020a893
Compare
| "x_opencti_description": description, | ||
| "x_opencti_score": score, | ||
| "x_opencti_labels": labels, | ||
| "x_opencti_create_indicator": self.config.flowtriq.create_indicator, |
| "x_opencti_description": source_desc, | ||
| "x_opencti_score": score, | ||
| "x_opencti_labels": labels + ["ddos:source"], | ||
| "x_opencti_create_indicator": self.config.flowtriq.create_indicator, |
| # Install Python modules | ||
| RUN pip install --upgrade pip && \ | ||
| pip install --no-cache-dir -r requirements.txt | ||
| RUN apk del git build-base |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Comment on lines
+94
to
+114
| # Filter by severity threshold and timestamp | ||
| current_state = self.helper.get_state() | ||
| last_incident_time = None | ||
| if current_state and "last_incident_time" in current_state: | ||
| last_incident_time = current_state["last_incident_time"] | ||
|
|
||
| processed = 0 | ||
| skipped_severity = 0 | ||
| skipped_seen = 0 | ||
| newest_incident_time = last_incident_time | ||
|
|
||
| for incident in incidents: | ||
| severity = incident.get("severity", "medium") | ||
| started_at = incident.get("started_at") | ||
|
|
||
| # Skip incidents already processed (by timestamp comparison) | ||
| if last_incident_time and started_at: | ||
| if started_at <= last_incident_time: | ||
| skipped_seen += 1 | ||
| continue | ||
|
|
Comment on lines
+139
to
+142
| # Track the newest incident timestamp for dedup state | ||
| if started_at: | ||
| if not newest_incident_time or started_at > newest_incident_time: | ||
| newest_incident_time = started_at |
Comment on lines
+7
to
+17
| # Copy the connector | ||
| COPY src /opt/flowtriq | ||
| WORKDIR /opt/flowtriq | ||
|
|
||
| # Install Python modules | ||
| RUN pip install --upgrade pip && \ | ||
| pip install --no-cache-dir -r requirements.txt | ||
| RUN apk del git build-base | ||
|
|
||
| # Expose and entrypoint | ||
| ENTRYPOINT ["python3", "main.py"] |
| connector: | ||
| id: 'ChangeMe' # Valid UUIDv4 | ||
| name: 'Flowtriq DDoS Incidents' | ||
| scope: 'flowtriq' # MIME type or SCO |
Comment on lines
+160
to
+162
| def create_incident_observable( | ||
| self, incident: dict | ||
| ) -> list[stix2.base._STIXBase]: |
- Parse timestamps to datetime for dedup comparison instead of raw strings - Remove unused entrypoint.sh (Dockerfile uses python3 directly) - Fix misleading scope comment in config.yml.sample - Optimize Dockerfile build deps with virtual packages - Fix TLP:CLEAR mapping and observable create_indicator flag
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new external-import connector that imports DDoS attack incident data from the Flowtriq network monitoring platform into OpenCTI as STIX 2.1 threat intelligence.
Closes #6821
What this connector does
GET /api/v1/incidentson a configurable schedule (default: hourly)Implementation
abuseipdb-ipblacklistconnector pattern exactly: Pydantic v2 settings viaconnectors-sdk,external_import_connectorpackage layout, standard DockerfileFiles
Test plan
python3 -m py_compileon all .py files)