Skip to content

RFC-016 Phase 1.5: OSINT collector executor, KG ingest, resolver wiring, and BGP #154

@rolandpg

Description

@rolandpg

Summary

Turn the RFC-016 OSINT scaffold from importable collectors into an end-to-end enrichment path that agents can run and persist into the knowledge graph.

Scope

  • Add a collector executor: input entity -> matching registry collectors -> validated CollectorTuple rows.
  • Persist collector output through KnowledgeGraph.add_node() / add_edge() using the existing kg_nodes / kg_edges storage model.
  • Wire zettelforge.osint.entity_resolver into OSINT ingest for canonical keys, dedupe, and last-write-wins property updates.
  • Promote the BGP collector from registered stub to functional passive enrichment, with mocked tests and fail-closed behavior.
  • Keep active port scanning gated behind explicit operator authorization; decide whether Phase 1.5 should implement it or leave it as documented stub.
  • Add user-facing docs for passive OSINT enrichment, optional dependencies, safety controls, and Python usage.

Acceptance Criteria

  • A Python API can run OSINT collection for a seed DomainName, IPv4Address, IPv6Address, ASNumber, or Netblock and persist resulting KG nodes/edges.
  • Collector results are ontology-validated before persistence.
  • Duplicate OSINT entities collapse through resolver canonicalization.
  • BGP collector is live for passive lookups or intentionally deferred with a narrower follow-up issue.
  • No tests touch the network; all live-source behavior is mocked.
  • Focused OSINT tests and the full regression suite pass.

Notes

Current baseline: RFC-016 Phase 1 is functional, with DNS / WHOIS-RDAP / crt.sh collectors and registered stubs for later phases. tests/test_osint_entities.py and tests/test_osint_collectors.py currently pass (109 passed).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions