Skip to content

Commit bdda997

Browse files
MatrixEditorMatrixEditor
committed
Add missing pypi definitions
1 parent 94f1a42 commit bdda997

2 files changed

Lines changed: 166 additions & 0 deletions

File tree

‎pypi/README.md‎

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
# Caterpillar - 🐛
2+
3+
[![python](https://img.shields.io/badge/Python-3.12+-blue?logo=python&logoColor=yellow)](https://www.python.org/downloads/)
4+
![![Latest Version](https://pypi.org/project/caterpillar-py/)](https://img.shields.io/github/v/release/MatrixEditor/caterpillar.svg?logo=github&label=Latest+Version)
5+
[![Build and Deploy Docs](https://github.com/MatrixEditor/caterpillar/actions/workflows/python-sphinx.yml/badge.svg)](https://github.com/MatrixEditor/caterpillar/actions/workflows/python-sphinx.yml)
6+
[![Run Tests](https://github.com/MatrixEditor/caterpillar/actions/workflows/python-test.yml/badge.svg)](https://github.com/MatrixEditor/caterpillar/actions/workflows/python-test.yml)
7+
![GitHub issues](https://img.shields.io/github/issues/MatrixEditor/caterpillar?logo=github)
8+
![GitHub License](https://img.shields.io/github/license/MatrixEditor/caterpillar?logo=github)
9+
10+
11+
Caterpillar is a Python 3.12+ library to pack and unpack structurized binary
12+
data (with support for 3.10+). It enhances the capabilities of [Python Struct](https://docs.python.org/3/library/struct.html)
13+
by enabling direct class declaration. More information about the different configuration
14+
options will be added in the future. Documentation is [here >](https://matrixeditor.github.io/caterpillar/).
15+
16+
*Caterpillar* is able to:
17+
18+
* Pack and unpack data just from processing Python class definitions (including support for powerful bitfields, c++-like templates and c-like unions!),
19+
* apply a wide range of data types (with endianess and architecture configuration),
20+
* dynamically adapt structs based on their inheritance layout,
21+
* reduce the used memory space using `__slots__`,
22+
* allowing you to place conditional statements into class definitions,
23+
* insert proper types into the class definition to support documentation and
24+
* it helps you to create cleaner and more compact code.
25+
* There is also a feature that lets you dynamically change the endian within a struct!
26+
* You can even extend Caterpillar and write your parsing logic in C or C++!!
27+
28+
> [!NOTE]
29+
> Python 3.14 breaks `with` statements in class definitions since `__annotations__` are added at the end
30+
> of a class definition. Therefore, `Digest` and conditional statements **ARE NOT SUPPORTED** using the `with` syntax in Python 3.14+.
31+
> As of version `2.4.5` the `Digest` class has a counterpart (`DigestField`), which can be used to manually specify a digest without
32+
> the need of a `ẁith` statement.
33+
34+
35+
## Give me some code!
36+
37+
```python
38+
from caterpillar.py import *
39+
40+
@bitfield(order=LittleEndian)
41+
class Header:
42+
version : 4 # 4bit integer
43+
valid : 1 # 1bit flag (boolean)
44+
ident : (8, CharFactory) # 8bit char
45+
# automatic alignment to 16bits
46+
47+
@struct(order=LittleEndian)
48+
class Format:
49+
magic : b"ITS MAGIC" # Supports string and byte constants directly
50+
header : Header
51+
a : uint8 # Primitive data types
52+
b : Dynamic + int32 # dynamic endian based on global config
53+
length : uint8 # String fields with computed lengths
54+
name : String(this.length) # -> you can also use Prefixed(uint8)
55+
56+
_hash_begin : DigestField.begin("hash", Md5_Algo) # custom actions, e.g. for hashes
57+
names : CString[uint8::] # Sequences with prefixed, computed lengths
58+
hash : Md5_Field("hash", verify=True) = None # automatic hash creation and verification + default value
59+
60+
# Instantiation (keyword-only arguments, magic is auto-inferred):
61+
obj = Format(
62+
header=Header(version=2, valid=True, ident="F"),
63+
a=1,
64+
b=2,
65+
length=3,
66+
name="foo",
67+
names=["a", "b"]
68+
)
69+
# Packing the object, reads as 'PACK obj FROM Format'
70+
# objects of struct classes can be packed right away
71+
blob = pack(obj, Format)
72+
# results in: b'ITS MAGIC0*\x01\x02\x00\x00\x00\x03foo\x02a\x00b\x00)\x9a...'
73+
74+
# Unpacking the binary data, reads as 'UNPACK Format FROM blob'
75+
obj2 = unpack(Format, blob)
76+
77+
# to pack with a different endian for field 'b', use _order
78+
data = pack(obj, Format, _order=BigEndian)
79+
```
80+
81+
This library offers extensive functionality beyond basic struct handling. For further details
82+
on its powerful features, explore the official [documentation](https://matrixeditor.github.io/caterpillar/),
83+
[examples](./examples/), and [test cases](./test/).
84+
85+
## Installation
86+
87+
> [!NOTE]
88+
> As of Caterpillar v2.1.2 it is possible to install the library without the need of
89+
> compiling the C extension.
90+
91+
### PIP installation (Python-only)
92+
93+
```bash
94+
pip install caterpillar-py
95+
```
96+
97+
### Python-only installation
98+
99+
```bash
100+
pip install "caterpillar[all]@git+https://github.com/MatrixEditor/caterpillar"
101+
```
102+
103+
### Installation + C-extension
104+
105+
```bash
106+
pip install "caterpillar[all]@git+https://github.com/MatrixEditor/caterpillar/#subdirectory=src/ccaterpillar"
107+
```
108+
109+
110+
## Starting Point
111+
112+
Please visit the [Documentation](https://matrixeditor.github.io/caterpillar/), it contains a complete tutorial on how to use this library.
113+
114+
## Other Approaches
115+
116+
A list of similar approaches to parsing structured binary data with Python can be taken from below:
117+
118+
* [construct](https://github.com/construct/construct)
119+
* [kaitai_struct](https://github.com/kaitai-io/kaitai_struct)
120+
* [hachoir](https://hachoir.readthedocs.io/en/latest/)
121+
* [mrcrowbar](https://github.com/moralrecordings/mrcrowbar)
122+
123+
The documentation also provides a [Comparison](https://matrixeditor.github.io/caterpillar/reference/introduction.html#comparison)
124+
to these approaches.
125+
126+
## License
127+
128+
Distributed under the GNU General Public License (V3). See [License](LICENSE) for more information.

‎pypi/pyproject.toml‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
[project]
2+
name = "caterpillar-py"
3+
version = "2.7.0"
4+
requires-python = ">=3.10"
5+
description="Library to pack and unpack structurized binary data."
6+
authors = [{ name = "MatrixEditor" }]
7+
maintainers = [{ name = "MatrixEditor" }]
8+
readme = "README.md"
9+
classifiers = [
10+
"Development Status :: 4 - Beta",
11+
"Intended Audience :: Developers",
12+
"Intended Audience :: Science/Research",
13+
14+
"License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
15+
16+
'Programming Language :: Python :: 3.10',
17+
'Programming Language :: Python :: 3.11',
18+
'Programming Language :: Python :: 3.12',
19+
'Programming Language :: Python :: 3.13',
20+
]
21+
dependencies = ["typing-extensions"]
22+
23+
[project.urls]
24+
"Homepage" = "https://github.com/MatrixEditor/caterpillar"
25+
"API-Docs" = "https://matrixeditor.github.io/caterpillar"
26+
27+
[project.optional-dependencies]
28+
# compression
29+
lzo = ["lzallright"]
30+
crypt = ["cryptography"]
31+
all = ["lzallright", "cryptography"]
32+
33+
[tool.setuptools.packages.find]
34+
where = ["../src"]
35+
include = ["caterpillar*"]
36+
37+
[tool.setuptools.package-data]
38+
caterpillar = ["include/**", ]

0 commit comments

Comments
 (0)