refactor(phishunt): use single open feed endpoint and fix related bugs (#6198)

[maximerafaillac]

Summary

Resolves #6198 — merge public/private feed logic into a single _process_feed method using the now-free feed.json endpoint.
Also resolves #6177 — the HTTP 403 caused by urllib missing a User-Agent is fixed by switching entirely to requests.
Also resolves #6593 — TLP:WHITE replaced with TLP:CLEAR.

Changes

Refactor

• Replace dual-path _process_public_feed / _process_private_feed with a single _process_feed consuming https://phishunt.io/feed.json
• Remove urllib, ssl, re imports — requests only
• Remove API key branching in process_message
• Deprecate PHISHUNT_API_KEY env var with a warning (same pattern as PHISHUNT_INTERVAL)

Bug fixes

• TLP:WHITE → TLP:CLEAR: use pycti.MarkingDefinition.generate_id('TLP', 'TLP:CLEAR') for a valid OpenCTI marking ID
• Duplicate STIX relationship ID: url→organization was using stix_domain.id in generate_id instead of stix_organization.id, causing the relationship to silently overwrite url→domain
• STIX pattern injection: escape \\ and ' in URL values before embedding in STIX patterns
• Invalid IPv4: validate IP with ipaddress.IPv4Address before creating the observable; skip IP/domain→ip/ip→location objects with a warning log when invalid, URL and domain are still ingested

Commits

Commit	Description
686f1fa	fix(phishunt): replace deprecated TLP:WHITE with TLP:CLEAR
26fbc36	refactor(phishunt): merge feeds into single _process_feed using feed.json
966e268	fix(phishunt): correct duplicate STIX ID on url→organization relationship
ea8dabf	fix(phishunt): escape backslashes and single quotes in STIX URL patterns
31a9565	fix(phishunt): validate IPv4 address before creating observable
cd5fc65	refactor(phishunt): deprecate PHISHUNT_API_KEY setting

Testing

• All pre-commit hooks pass (black, flake8, isort)
• Existing settings tests updated to remove api_key references
• New test case added: deprecated api_key is accepted without error

[codecov]

Codecov Report

❌ Patch coverage is 96.96970% with 1 line in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
...xternal-import/phishunt/src/connector/connector.py	96.66%	1 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (88ccaf1) and HEAD (6ee8999). Click for more details.

HEAD has 122 uploads less than BASE

Flag	BASE (88ccaf1)	HEAD (6ee8999)
connectors	126	4

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #6774       +/-   ##
===========================================
- Coverage   34.25%    0.32%   -33.94%     
===========================================
  Files        2007     1915       -92     
  Lines      123209   120577     -2632     
===========================================
- Hits        42206      390    -41816     
- Misses      81003   120187    +39184     

Files with missing lines	Coverage Δ
external-import/phishunt/src/connector/settings.py	87.09% <100.00%> (+0.43%)	⬆️
...xternal-import/phishunt/src/connector/connector.py	73.07% <96.66%> (+22.29%)	⬆️

... and 1153 files with indirect coverage changes

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

Thank you for your contribution, but we need you to sign your commits. Please see https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits

[Copilot]

Pull request overview

This PR refactors the Phishunt external-import connector to consume the now-public https://phishunt.io/feed.json via requests, removes the legacy public/private feed split, and addresses several ingestion correctness issues (TLP marking, deterministic relationship IDs, STIX pattern escaping, and IPv4 validation). It also updates tests and container packaging accordingly.

Changes:

• Consolidate feed handling into a single _process_feed() using requests.get(feed.json) and remove API-key branching.
• Update ingested STIX objects/relationships: TLP:WHITE → TLP:CLEAR, fix relationship ID generation, escape URL values in STIX patterns, and skip invalid IPv4s.
• Update config/settings migration and tests for deprecated PHISHUNT_API_KEY, and adjust Dockerfile entrypoint.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
external-import/phishunt/src/connector/connector.py	Implements unified feed ingestion, new TLP:CLEAR marking, relationship fixes, pattern escaping, IPv4 validation, and bundle sending.
external-import/phishunt/src/connector/settings.py	Removes api_key from config model and warns when legacy PHISHUNT_API_KEY is provided.
external-import/phishunt/tests/test_main.py	Adds _process_feed unit tests (including HTTP error handling, invalid IP behavior, and indicator toggling).
external-import/phishunt/tests/tests_connector/test_settings.py	Updates settings validation cases and accepts legacy api_key input without failing validation.
external-import/phishunt/Dockerfile	Changes runtime entrypoint to run python3 main.py from the connector workdir.
external-import/phishunt/docker-compose.yml	Simplifies the sample OPENCTI_TOKEN placeholder.