Skip to content

Commit be4565b

Browse files
EdgeTypEEdgeTypE
committed
Revise and expand documentation for PatternLab
Major updates to documentation files: README, API reference, getting started, configuration, index, plugin developer guide, and test reference. Changes include improved English language coverage, expanded examples, clearer explanations of plugin architecture, configuration usage, and API integration. Documentation is now more accessible for new users and developers, with concise guides for installation, usage, plugin development, and test reference.
1 parent 30a8e82 commit be4565b

9 files changed

Lines changed: 608 additions & 887 deletions

File tree

‎README.md‎

Lines changed: 75 additions & 155 deletions
Original file line numberDiff line numberDiff line change
@@ -1,200 +1,120 @@
11
# PatternLab
22

3-
Binary pattern analysis framework for statistical testing and transformation of binary data.
3+
PatternLab is a comprehensive, plugin-based framework for binary data analysis in Python. It provides a powerful engine to apply statistical tests, cryptographic analysis, and structural format detection on any binary data source.
44

55
## Features
66

7-
- **Plugin Architecture**: Extensible system for transforms and statistical tests
8-
- **XOR Transform**: Apply constant XOR transformation to binary data
9-
- **Monobit Test**: Statistical frequency test for binary sequences
10-
- **CLI Interface**: Command-line tool for easy analysis
11-
- **Python API**: Programmatic access to all functionality
7+
- **Extensible Plugin Architecture**: Easily add new statistical tests, data transformers, or visualizers.
8+
- **Rich Plugin Library**: Comes with a wide range of built-in plugins for:
9+
- **Statistical Analysis**: NIST-like tests (Monobit, Runs, FFT), Dieharder-inspired tests, and advanced metrics like Hurst Exponent and Entropy.
10+
- **Cryptographic Analysis**: Detects ECB mode encryption, repeating-key XOR patterns, and searches for known constants like AES S-boxes.
11+
- **Structural Analysis**: Basic parsers for formats like ZIP, PNG, and PDF.
12+
- **Machine Learning**: Anomaly detection using Autoencoders, LSTMs, and pre-trained classifiers.
13+
- **Multiple Interfaces**: Use PatternLab the way you want:
14+
- **Command-Line Interface (CLI)** for scripting and automation.
15+
- **Web User Interface (Streamlit)** for interactive analysis and visualization.
16+
- **Text-based User Interface (TUI)** for terminal-based interaction.
17+
- **REST API (FastAPI)** to integrate PatternLab into other services.
18+
- **High-Performance Engine**: Supports parallel test execution, streaming analysis for large files, and sandboxed plugin execution for security and stability.
1219

1320
## Installation
1421

15-
### From Source
22+
It is recommended to install PatternLab in a virtual environment.
1623

1724
```bash
1825
# Clone the repository
19-
git clone <repository-url>
20-
cd patternlab
26+
git clone https://github.com/your-username/pattern-analyzer.git
27+
cd pattern-analyzer
2128

22-
# Install in development mode
23-
pip install -e .
24-
```
25-
26-
### Dependencies
29+
# Create and activate a virtual environment
30+
python -m venv .venv
31+
# On Windows: .venv\Scripts\activate
32+
# On macOS/Linux: source .venv/bin/activate
2733

28-
- Python 3.10+
29-
- click (for CLI)
30-
- pytest (for testing)
34+
# Install the package in editable mode with all optional dependencies
35+
pip install -e .[test,ml,ui]
36+
```
37+
The optional dependencies are:
38+
- `test`: for running the test suite with `pytest`.
39+
- `ml`: for machine learning-based plugins (TensorFlow, scikit-learn).
40+
- `ui`: for the Streamlit web UI and Textual TUI.
3141

32-
## Usage
42+
## Quick Start
3343

34-
### Command Line Interface
44+
### Command Line Interface (CLI)
3545

36-
Analyze a binary file:
46+
Analyze a binary file using a default set of tests and save the report.
3747

3848
```bash
39-
# Basic analysis (writes JSON with top-level keys "results" and "scorecard")
40-
patternlab analyze input.bin --out results.json
41-
42-
# With XOR constant transformation
43-
patternlab analyze input.bin --out results.json --xor-value 255
49+
patternlab analyze test.bin --out report.json
4450
```
4551

46-
Options:
47-
- `input-file`: Path to binary file to analyze
48-
- `--out`: Output JSON file (default: report.json)
49-
- `--xor-value`: XOR value for transformation (0-255, default: 0)
50-
51-
CLI example — updated JSON output (file written to `results.json`; top-level schema shown below):
52-
53-
Top-level schema:
54-
55-
```json
56-
{
57-
"results": [...],
58-
"scorecard": {...}
59-
}
60-
```
52+
Use a specific configuration profile for a focused analysis (e.g., cryptographic tests).
6153

62-
Full example output (file written to `results.json`):
63-
64-
```json
65-
{
66-
"results": [
67-
{
68-
"test_name": "monobit",
69-
"passed": true,
70-
"p_value": 0.05,
71-
"p_values": {"overall": 0.05},
72-
"effect_sizes": {"overall": 0.8},
73-
"flags": [],
74-
"metrics": {"bit_count": 1000},
75-
"z_score": 1.96,
76-
"evidence": null,
77-
"fdr_rejected": false,
78-
"fdr_q": 0.05
79-
}
80-
],
81-
"scorecard": {
82-
"failed_tests": 0,
83-
"mean_effect_size": 0.8,
84-
"p_value_distribution": {
85-
"count": 1,
86-
"mean": 0.05,
87-
"median": 0.05,
88-
"stdev": 0.0,
89-
"histogram": {"0-0.01": 0, "0.01-0.05": 0, "0.05-0.1": 1, "0.1-1.0": 0}
90-
},
91-
"total_tests": 1,
92-
"fdr_q": 0.05
93-
}
94-
}
54+
```bash
55+
patternlab analyze encrypted.bin --profile crypto --out crypto_report.json
9556
```
9657

9758
### Python API
9859

60+
Programmatically run an analysis pipeline.
61+
9962
```python
10063
from patternlab.engine import Engine
101-
from patternlab.plugins.xor_const import XOPlugin
102-
from patternlab.plugins.monobit import MonobitTest
103-
from patternlab.plugin_api import BytesView, serialize_testresult
104-
import json
10564

106-
# Engine ve plugin kayıtları
65+
# Initialize the analysis engine
10766
engine = Engine()
108-
engine.register_transform('xor_const', XOPlugin())
109-
engine.register_test('monobit', MonobitTest())
110-
111-
# Veri analizi
112-
with open('input.bin', 'rb') as f:
113-
data = f.read()
11467

115-
config = {'transforms': [{'name': 'xor_const', 'params': {'xor_value': 255}}], 'tests': [{'name': 'monobit', 'params': {}}]}
116-
output = engine.analyze(data, config)
68+
# Load data from a file
69+
with open("test.bin", "rb") as f:
70+
data_bytes = f.read()
11771

118-
# `engine.analyze` fonksiyonu top-level bir dict döner:
119-
# { "results": [...], "scorecard": {...} }
120-
# TestResult'ları canonical JSON şemasına çevirmek için serialize_testresult kullanılmaktadır.
121-
print(json.dumps(output, indent=2))
122-
```
123-
124-
Örnek serializasyon (yeni TestResult şeması — CLI ve engine tarafından üretilen her bir sonuç bu şemaya uyar):
125-
```json
126-
{
127-
"test_name": "monobit",
128-
"passed": true,
129-
"p_value": 0.05,
130-
"p_values": {"overall": 0.05},
131-
"effect_sizes": {"overall": 0.8},
132-
"flags": ["flag1", "flag2"],
133-
"metrics": {"metric1": 123, "metric2": 456},
134-
"z_score": 1.96,
135-
"evidence": "Some evidence string",
136-
"fdr_rejected": false,
137-
"fdr_q": 0.05
72+
# Define an analysis configuration
73+
# This example applies a simple XOR transform before running the monobit test
74+
config = {
75+
"transforms": [{"name": "xor_const", "params": {"xor_value": 127}}],
76+
"tests": [{"name": "monobit", "params": {}}],
77+
"fdr_q": 0.05 # Set the False Discovery Rate significance level
13878
}
139-
```
14079

141-
Not: Eğer eski `details` alanı varsa, `serialize_testresult` fonksiyonu onu `metrics` içine birleştirir.
80+
# Run the analysis
81+
output = engine.analyze(data_bytes, config)
14282

143-
## Available Plugins
144-
145-
### Transform Plugins
146-
147-
- **xor_const**: Applies XOR transformation with constant value
148-
149-
### Test Plugins
150-
151-
- **monobit**: Monobit frequency test for random number generators
152-
153-
## Testing
154-
155-
Run the test suite:
156-
157-
```bash
158-
# Run all tests
159-
pytest
160-
161-
# Run with verbose output
162-
pytest -v
163-
164-
# Run specific test file
165-
pytest tests/test_xor.py
83+
# Print the results
84+
import json
85+
print(json.dumps(output, indent=2))
16686
```
16787

16888
## Project Structure
16989

17090
```
171-
patternlab/
172-
├── __init__.py # Package initialization
173-
├── plugin_api.py # Plugin base classes and data structures
174-
├── engine.py # Main analysis engine
175-
├── cli.py # Command line interface
176-
└── plugins/
177-
├── __init__.py
178-
├── xor_const.py # XOR transform plugin
179-
└── monobit.py # Monobit test plugin
180-
tests/
181-
├── test_xor.py
182-
└── test_monobit.py
91+
pattern-analyzer/
92+
├── patternlab/ # Main source code for the framework
93+
│ ├── plugins/ # Built-in analysis and transform plugins
94+
│ ├── __init__.py
95+
│ ├── engine.py # The core analysis engine
96+
│ ├── plugin_api.py # Base classes for plugins (Test, Transform, Visual)
97+
│ ├── cli.py # Click-based Command Line Interface
98+
│ ├── api.py # FastAPI-based REST API
99+
│ ├── tui.py # Textual-based Terminal User Interface
100+
│ └── ...
101+
├── app.py # Streamlit Web User Interface
102+
├── docs/ # Documentation files for MkDocs
103+
├── tests/ # Pytest unit and integration tests
104+
├── pyproject.toml # Project metadata and dependencies
105+
└── README.md
183106
```
184107

185-
## License
186-
187-
MIT License - see LICENSE file for details.
188-
189108
## Contributing
190109

191-
1. Fork the repository
192-
2. Create a feature branch
193-
3. Add tests for new functionality
194-
4. Ensure all tests pass: `pytest`
195-
5. Submit a pull request
110+
Contributions are welcome! Please feel free to open an issue or submit a pull request.
196111

197-
## Requirements
112+
1. Fork the repository.
113+
2. Create a new feature branch (`git checkout -b feature/my-new-feature`).
114+
3. Implement your changes and add tests.
115+
4. Ensure all tests pass (`pytest`).
116+
5. Submit a pull request.
117+
118+
## License
198119

199-
- Python 3.10 or higher
200-
- No external dependencies beyond standard library (except for development)
120+
This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.

0 commit comments

Comments
 (0)