Skip to content

Commit 8963bc9

Browse files
authored
Merge pull request #12 from malx-labs/feature/plugin-eco
v0.4.0 release - Feature/plugin eco into main
2 parents 320f3ff + 1952e7a commit 8963bc9

33 files changed

Lines changed: 2540 additions & 174 deletions

CONTRIBUTING.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Welcome
22

3-
Thank you for your interest in contributing to malx‑ioc‑extractor. This project is part of the MalX Labs ecosystem — a collection of modern, developer‑friendly security tools focused on safe, scalable analysis.
3+
Thank you for your interest in contributing to IOCX. This project is part of the MalX Labs ecosystem — a collection of modern, developer‑friendly security tools focused on safe, scalable analysis.
44

55
We appreciate improvements of all kinds — bug fixes, new extractors, documentation updates, and thoughtful discussions.
66

@@ -27,7 +27,7 @@ Open an issue or submit a PR with a clear description.
2727

2828
### Add new IOC detectors
2929

30-
Regex‑based detectors live under `extractors/`.
30+
Regex‑based detectors live under `detectors/extractors/`.
3131
Please include:
3232

3333
- a clear regex
@@ -55,7 +55,7 @@ Better examples, diagrams, or explanations are always appreciated.
5555
Create your own fork and clone it locally:
5656
```bash
5757

58-
git clone https://github.com/malx-labs/malx-ioc-extractor.git
58+
git clone https://github.com/iocx-dev/iocx.git
5959

6060
```
6161

@@ -148,7 +148,7 @@ We use pytest for the test suite.
148148
Extractor modules live in:
149149
```plaintext
150150
151-
iocx/extractors/
151+
iocx/detectors/extractors/
152152
153153
```
154154

README-pypi.md

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# IOCX — Static IOC Extraction Engine
2+
3+
IOCX is a fast, safe, deterministic engine for extracting Indicators of Compromise (IOCs) from binaries, text, and logs.
4+
It performs **pure static analysis** — no execution, no sandboxing, no risk.
5+
6+
## Features
7+
8+
- Extracts IOCs from Windows PE files and raw text
9+
- Detects URLs, domains, IPv4/IPv6, file paths, hashes, emails, Base64
10+
- Crypto wallet detection (Ethereum, Bitcoin)
11+
- Deterministic output suitable for automation
12+
- Minimal dependencies and safe for enterprise environments
13+
- CLI and Python API
14+
15+
## Installation
16+
17+
```bash
18+
pip install iocx
19+
```
20+
21+
## CLI Usage
22+
23+
```bash
24+
iocx suspicious.exe
25+
```
26+
27+
```bash
28+
echo "Visit http://bad.example.com" | iocx -
29+
```
30+
31+
## Python API
32+
33+
```python
34+
from iocx.engine import Engine
35+
36+
engine = Engine()
37+
results = engine.extract("suspicious.exe")
38+
print(results)
39+
```
40+
41+
## Why IOCX?
42+
43+
- Static‑only design (never executes untrusted code)
44+
- Binary‑aware IOC extraction
45+
- Stable JSON schema
46+
- High performance (~200 MB/s throughput)
47+
- Ideal for DFIR, SOC automation, CI/CD, and threat‑intel pipelines
48+
49+
## Extensibility
50+
51+
IOCX includes a lightweight plugin system that allows you to add custom detectors, parsers, and transformation rules.
52+
Plugins can emit new IOC categories, override built-in behaviour, or integrate IOCX into larger analysis pipelines.
53+
54+
See the documentation for details on writing detectors and plugins.
55+
56+
## License
57+
58+
MIT License

README.md

Lines changed: 70 additions & 110 deletions
Original file line numberDiff line numberDiff line change
@@ -3,88 +3,97 @@
33
<img src="https://img.shields.io/pypi/v/iocx?logo=pypi&logoColor=white" alt="PyPI Version">
44
</a>
55
<img src="https://img.shields.io/badge/coverage-97%25-brightgreen" alt="Coverage">
6-
<img src="https://img.shields.io/badge/tests-273_passed-brightgreen" alt="Tests">
6+
<img src="https://img.shields.io/badge/tests-313_passed-brightgreen" alt="Tests">
77
<img src="https://img.shields.io/badge/python-3.12-blue" alt="Python Version">
8-
<a href="https://github.com/malx-labs/malx-ioc-extractor/blob/main/LICENSE">
9-
<img src="https://img.shields.io/github/license/malx-labs/malx-ioc-extractor" alt="License">
8+
<a href="https://github.com/iocx-dev/iocx/blob/main/LICENSE">
9+
<img src="https://img.shields.io/github/license/iocx-dev/iocx" alt="License">
1010
</a>
11-
<a href="https://github.com/malx-labs/malx-ioc-extractor/actions">
12-
<img src="https://img.shields.io/github/actions/workflow/status/malx-labs/malx-ioc-extractor/ci.yml?label=build" alt="Build Status">
11+
<a href="https://github.com/iocx-dev/iocx/actions">
12+
<img src="https://img.shields.io/github/actions/workflow/status/iocx-dev/iocx/ci.yml?label=build" alt="Build Status">
1313
</a>
1414
<img src="https://img.shields.io/badge/v0.2.0_performance-1MB_in_0.0053s-brightgreen" alt="Performance">
1515
<img src="https://img.shields.io/badge/v0.2.0_throughput-~200MB%2Fs-brightgreen" alt="Throughput">
1616
<img src="https://img.shields.io/badge/v0.2.0_pathological_IPv6-0.0005s-brightgreen" alt="Pathological IPv6 Timing">
1717
</p>
1818

19-
# malx‑ioc‑extractor
19+
# IOCX — Static IOC Extraction for Binaries, Text, and Artifacts
2020

21-
**Static IOC extraction for binaries, text, and artifacts — fast, safe, and open‑source.**
21+
**Fast, safe, deterministic IOC extraction for DFIR, SOC automation, and large-scale threat analysis.**
2222

23-
malx‑ioc‑extractor is a lightweight, extensible engine for extracting Indicators of Compromise (IOCs) using pure static analysis. No execution. No sandboxing. No risk.
24-
Built for DFIR workflows, SOC automation, and large‑scale threat analysis.
23+
IOCX is a lightweight, extensible engine for extracting Indicators of Compromise (IOCs) using **pure static analysis**. No execution. No sandboxing. No risk.
2524

26-
It’s designed to be:
25+
Built for:
2726

28-
- **Safe** — never executes untrusted code
29-
- **Fast** — built for automation and pipelines
30-
- **Extensible** — plug in your own regexes, parsers, and rules
31-
- **Developer‑friendly** — clean API, CLI, and examples
32-
- **Open‑source** — the extraction engine is free; enrichment lives in the MalX cloud platform
27+
- DFIR workflows
28+
- SOC automation
29+
- Threat-intel pipelines
30+
- CI/CD security checks
31+
- Large‑scale batch processing
3332

3433
This project is the foundation of the MalX Labs ecosystem for scalable, modern threat‑analysis tooling.
3534

36-
## Why malx‑ioc‑extractor?
35+
## Why IOCX?
3736

38-
malx‑ioc‑extractor is designed for environments where safety, determinism, and automation matter. While many IOC extractors operate only on raw text, malx‑ioc‑extractor includes binary‑aware static analysis and an extensible rule system, making it suitable for DFIR pipelines, CI systems, and high‑volume threat‑intel processing.
37+
IOCX is designed for environments where **safety, determinism, and automation** matter. Unlike extractors that operate only on raw text, IOCX includes **binary‑aware static analysis**, a **plugin-friendly rule system**, and a **stable JSON schema**.
3938

40-
**Key advantages**
39+
### Key advantages
4140

42-
- **Static‑only design**no execution, no sandboxing, and no risk of running untrusted code
43-
- **Binary parsing** — extracts indicators from Windows PE files in addition to raw text
44-
- **Deterministic behaviour** — stable output and predictable performance, ideal for automated workflows
45-
- **Extensible rule engine**plug in custom detectors, parsers, and enrichment logic
46-
- **Consistent JSON schema**uniform output that integrates cleanly with SIEM, SOAR, and log pipelines
47-
- **Low dependency footprint**minimal attack surface and safe for enterprise environments
48-
- **Designed for pipelines** — fast start‑up, fast throughput, and no heavyweight runtime requirements
41+
- **Static‑only design**never executes untrusted code
42+
- **Binary parsing** — extracts IOCs from Windows PE files in addition to raw text
43+
- **Deterministic behaviour** — stable output and predictable performance, ideal for pipelines
44+
- **Extensible rule engine** — custom detectors, parsers, and plugins
45+
- **Consistent JSON schema**clean integration with SIEM/SOAR
46+
- **Low dependency footprint** — safe for enterprise environments
47+
- **Pipeline-ready** — fast start‑up, fast throughput
4948

50-
## Use Cases
49+
---
50+
51+
## What IOCX *Is Not*
52+
53+
To avoid confusion:
5154

52-
malx‑ioc‑extractor fits naturally into DFIR, security automation, and threat‑intelligence workflows. Typical usage patterns include:
55+
- Not a sandbox
56+
- Not a malware emulator
57+
- Not a behavioural analysis tool
58+
- Not an enrichment engine (that lives in the MalX Cloud platform)
59+
60+
IOCX is **static extraction only**, by design.
61+
62+
## Use Cases
5363

5464
### SOC & Incident Response
55-
- Extract indicators from suspicious emails, alerts, or analyst clipboard text
56-
- Parse IOCs from incident reports and triage notes into structured JSON
57-
- Safely inspect malware samples statically without executing anything
65+
- Extract indicators from emails, alerts, or analyst clipboard text
66+
- Parse IOCs from reports into structured JSON
67+
- Safely inspect malware samples without execution
5868

5969
### Threat Intelligence Processing
60-
- Normalize indicators from threat‑intel feeds
61-
- Batch‑process dumps of unstructured text into machine‑readable IOC sets
62-
- Build enrichment pipelines on top of the deterministic output format
70+
- Normalize indicators from feeds
71+
- Batch‑process unstructured text
72+
- Build enrichment pipelines on top of deterministic output
6373

6474
### CI/CD & DevSecOps
65-
- Scan new binaries for embedded indicators before publishing artifacts
66-
- Integrate IOC extraction into automated security checks
67-
- Detect accidental inclusion of URLs or addresses during build steps
75+
- Scan binaries for embedded indicators before publishing
76+
- Integrate IOC extraction into automated checks
77+
- Detect accidental inclusion of URLs or addresses in builds
6878

6979
### Bulk Automation & Scripting
70-
- Pipe logs, artifacts, or telemetry through malx‑ioc‑extractor to extract actionable indicators
71-
- Use the Python API for batch workflows, ETL pipelines, or custom tooling
72-
- Combine with rule extensions to tailor detection to internal patterns or datasets
80+
- Pipe logs or artifacts through IOCX
81+
- Use the Python API for ETL or batch workflows
82+
- Extend with custom detectors for internal patterns
7383

74-
## v0.3.0 — Stronger Architecture, New Crypto IOC Detection, Same Blazing Performance
84+
## Version Highlights
7585

76-
Version 0.3.0 is a foundational release that tightens the engine’s architecture, expands IOC coverage with Ethereum and Bitcoin wallet detection, and preserves the extractor’s extremely fast performance even on multi‑megabyte inputs. This release makes the engine more predictable, extensible, and resilient for the long-term extensibility.
86+
### v0.3.0 — Stronger Architecture, New Crypto IOC Detection
7787

78-
## v0.2.0 — High‑Reliability IP Detection in Hostile Data
88+
- Ethereum & Bitcoin wallet detection
89+
- Improved architecture for long-term extensibility
90+
- Same blazing performance on multi-MB inputs
7991

80-
Version 0.2.0 significantly improves IPv4/IPv6 extraction in noisy, malformed, mixed-content environments — the kind often seen in:
92+
### v0.2.0 — High‑Reliability IP Detection
8193

82-
- SIEM log lines
83-
- network captures
84-
- DFIR corpus samples
85-
- pasted analyst dumps
94+
Significant improvements to IPv4/IPv6 extraction in noisy, malformed, mixed-content environments
8695

87-
### Real CLI Output (Chaos Corpus Sample)
96+
## Real CLI Output (Chaos Corpus Sample)
8897

8998
```json
9099
$ iocx chaos_corpus.json
@@ -165,26 +174,21 @@ All measurements from the latest performance suite:
165174

166175
| Sample Type | Time |
167176
|------------------------------|----------|
168-
| IP | |
169-
|==============================|==========|
177+
| **IP** | |
170178
| 1 MB mixed‑content sample | 0.0070s |
171179
| Pathological IPv6 blob | 0.0004s |
172180
| 100 KB sample | 0.0008s |
173181
| 300 KB sample | 0.0021s |
174182
| 600 KB sample | 0.0038s |
175183
| 1 MB sample | 0.0068s |
176-
|------------------------------|----------|
177-
| Filepath | |
178-
|==============================|==========|
184+
| **Filepath** | |
179185
| 1 MB mixed‑content sample | 0.0040s |
180186
| Pathological deep unix path | 0.0237s |
181187
| 300 KB sample | 0.0011s |
182188
| 600 KB sample | 0.0022s |
183189
| 1000 KB sample | 0.0038s |
184190
| 1500 KB sample | 0.0055s |
185-
|------------------------------|----------|
186-
| Crypto | |
187-
|==============================|==========|
191+
| **Crypto** | |
188192
| 1 MB mixed‑content sample | 0.0021s |
189193
| Pathological ETH-like blob | 0.0012s |
190194
| 300 KB sample | 0.0006s |
@@ -217,7 +221,7 @@ All measurements from the latest performance suite:
217221
- Hashes (MD5 / SHA1 / SHA256 / SHA512 / Generic Hex)
218222
- Email addresses
219223
- Base64
220-
- Crypto wallets (Ethereum, Bitcoin) (new in v0.3.0)
224+
- Crypto wallets (Ethereum / Bitcoin)
221225

222226
### Static PE Parsing
223227

@@ -338,64 +342,28 @@ print(results)
338342
## Architecture
339343
```plaintext
340344
341-
malx-ioc-extractor/
345+
iocx/
342346
343347
├── examples/ # Sample files + generators
344-
├── tests/ # Unit and integration tests
348+
├── docs/ # Detector contracts, overlap suppression rules, and plugin authoring guidelines
349+
├── tests/ # Unit, integration, fuzz, robustness, and performance tests
345350
├── iocx
346351
├── detectors/ # Regex-based IOC detectors
347352
├── parsers/ # PE parsing, string extraction
353+
├── plugins/ # Plugin API and registry
348354
├── cli/ # Command-line interface
349355
350356
```
351357

352358
The engine is intentionally modular so components can be extended or replaced easily.
353359

354-
## Extending the Engine
355-
356-
You can add custom:
357-
358-
- Regex detectors
359-
- File parsers
360-
- Normalisation logic
361-
362-
### Register a custom detector
363-
As an example, the following detector extracts the word "SECRET" from a string of text:
364-
```python
365-
366-
import re
367-
from iocx.models import Detection
368-
from iocx.detectors.registry import register_detector
369-
370-
SECRET_REGEX = re.compile(r"\bSECRET\b")
371-
372-
@register_detector("custom.secret")
373-
def detect_secret(text: str):
374-
results = []
375-
for m in SECRET_REGEX.finditer(text):
376-
results.append(
377-
Detection(
378-
value=m.group(0),
379-
start=m.start(),
380-
end=m.end(),
381-
category="custom.secret"
382-
)
383-
)
384-
return results
385-
386-
```
387-
Assuming the detector was saved as `my_detector.py`, consume the custom detector - if the detector fires, its results will be added to the list of detected IOCs:
388-
```python
389-
390-
from iocx.engine import Engine
391-
import my_detector
392-
393-
engine = Engine()
360+
## Extending IOCX
394361

395-
results = engine.extract("This is a SECRET message.")
396-
print(results)
362+
See `docs/specs/` for:
397363

398-
```
364+
- Detector contracts
365+
- Overlap suppression rules
366+
- Plugin authoring guidelines
399367

400368
## Safe Testing (No Malware Required)
401369

@@ -423,14 +391,6 @@ See CONTRIBUTING.md for full guidelines.
423391
If you discover a security issue, do not open a GitHub issue.
424392
Please follow the instructions in SECURITY.md.
425393

426-
## Related Projects (MalX Labs)
427-
428-
- malx-core — foundational primitives
429-
- malx-utils — shared utilities
430-
- malx-sandbox — dynamic analysis environment
431-
- malx-forge — adversarial payload tooling
432-
- malx-archive — research + PoCs
433-
434394
## License
435395

436396
Licensed under the MIT License. See LICENSE for details.

SECURITY.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Security Policy
22

3-
Thank you for your interest in the security of malx-ioc-extractor.
3+
Thank you for your interest in the security of IOCX.
44
We take security seriously and aim to provide a trustworthy, minimal‑dependency tool for static IOC extraction across binaries, text, and logs.
55

66
This document explains how we handle security, how to report vulnerabilities, and what you can expect from us.

0 commit comments

Comments
 (0)