Skip to content

Commit 630418d

Browse files
PromiseFruPromiseFru
authored
Merge pull request #109 from smswithoutborders/staging
Staging
2 parents 97fdfe2 + 80e40bc commit 630418d

2 files changed

Lines changed: 249 additions & 12 deletions

File tree

‎docs/specification.md‎

Lines changed: 242 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,10 +6,14 @@
66
- [Content Format V0](#content-format-v0)
77
- [Content Format V1](#content-format-v1)
88
- [Content Format V2](#content-format-v2)
9+
- [Content Format V3](#content-format-v3)
10+
- [Content Format V4](#content-format-v4)
911
- [Payload Format](#supported-payload-versions)
1012
- [Payload Format V0](#payload-format-v0)
1113
- [Payload Format V1](#payload-format-v1)
1214
- [Payload Format V2](#payload-format-v2)
15+
- [Payload Format V3](#payload-format-v3)
16+
- [Payload Format V4](#payload-format-v4)
1317

1418
## Content Format
1519

@@ -182,13 +186,150 @@
182186
- **2 byte**: Length of `refresh_token` field (set to `0`).
183187
- **Variable**: Value of `from` field (considered the test ID).
184188

189+
### Content Format V3
190+
191+
> [!NOTE]
192+
>
193+
> For detailed instructions on encrypting the content format using the Double Ratchet algorithm, refer to the [smswithoutborders_lib_sig documentation](https://github.com/smswithoutborders/lib_signal_double_ratchet_python?tab=readme-ov-file#double-ratchet-implementations).
194+
195+
> [!NOTE]
196+
>
197+
> This format improves efficiency by only including length fields for applicable fields per service type, rather than including all fields with zero lengths.
198+
>
199+
> **All 2-byte length fields are encoded as unsigned little-endian.**
200+
201+
1. **Email format**: Binary-encoded fields with the following structure:
202+
203+
- **1 byte**: Length of `from` field.
204+
- **2 bytes**: Length of `to` field.
205+
- **2 bytes**: Length of `cc` field.
206+
- **2 bytes**: Length of `bcc` field.
207+
- **1 byte**: Length of `subject` field.
208+
- **2 bytes**: Length of `body` field.
209+
- **2 bytes**: Length of `access_token` field (optional).
210+
- **2 bytes**: Length of `refresh_token` field (optional).
211+
- **Variable**: Value of `from` field.
212+
- **Variable**: Value of `to` field.
213+
- **Variable**: Value of `cc` field.
214+
- **Variable**: Value of `bcc` field.
215+
- **Variable**: Value of `subject` field.
216+
- **Variable**: Value of `body` field.
217+
- **Variable**: Value of `access_token` field (if present).
218+
- **Variable**: Value of `refresh_token` field (if present).
219+
220+
2. **Text format**: Binary-encoded fields with the following structure:
221+
222+
- **1 byte**: Length of `from` field.
223+
- **2 bytes**: Length of `body` field.
224+
- **2 bytes**: Length of `access_token` field (optional).
225+
- **2 bytes**: Length of `refresh_token` field (optional).
226+
- **Variable**: Value of `from` field.
227+
- **Variable**: Value of `body` field.
228+
- **Variable**: Value of `access_token` field (if present).
229+
- **Variable**: Value of `refresh_token` field (if present).
230+
231+
3. **Message format**: Binary-encoded fields with the following structure:
232+
233+
- **1 byte**: Length of `from` field.
234+
- **2 bytes**: Length of `to` field.
235+
- **2 bytes**: Length of `body` field.
236+
- **Variable**: Value of `from` field.
237+
- **Variable**: Value of `to` field.
238+
- **Variable**: Value of `body` field.
239+
240+
4. **Test format**: Binary-encoded fields with the following structure:
241+
242+
- **1 byte**: Length of `from` field.
243+
- **Variable**: Value of `from` field (considered the test ID).
244+
245+
### Content Format V4
246+
247+
> [!NOTE]
248+
>
249+
> For detailed instructions on encrypting the content format using the Double Ratchet algorithm, refer to the [smswithoutborders_lib_sig documentation](https://github.com/smswithoutborders/lib_signal_double_ratchet_python?tab=readme-ov-file#double-ratchet-implementations).
250+
251+
> [!NOTE]
252+
>
253+
> This format extends V3 by adding support for image content. A content type indicator determines which content types are present.
254+
>
255+
> **All 2-byte length fields are encoded as unsigned little-endian.**
256+
257+
#### Content Type Indicator
258+
259+
The content type indicator is a single byte with the following values:
260+
261+
- **0**: Text-only content
262+
- **1**: Image-only content
263+
- **2**: Text and image content
264+
265+
1. **Email format**: Binary-encoded fields with the following structure:
266+
267+
- **1 byte**: Content type indicator
268+
- **1 byte**: Length of `from` field (if text content present).
269+
- **2 bytes**: Length of `to` field (if text content present).
270+
- **2 bytes**: Length of `cc` field (if text content present).
271+
- **2 bytes**: Length of `bcc` field (if text content present).
272+
- **1 byte**: Length of `subject` field (if text content present).
273+
- **2 bytes**: Length of `body` field (if text content present).
274+
- **2 bytes**: Length of `access_token` field (if text content present and optional).
275+
- **2 bytes**: Length of `refresh_token` field (if text content present and optional).
276+
- **2 bytes**: `Image Session ID` (if image content present).
277+
- **1 byte**: `Image Segment info` (if image content present).
278+
- **2 bytes**: Length of `Image` (if image content present).
279+
- **Variable**: Value of `from` field (if text content present).
280+
- **Variable**: Value of `to` field (if text content present).
281+
- **Variable**: Value of `cc` field (if text content present).
282+
- **Variable**: Value of `bcc` field (if text content present).
283+
- **Variable**: Value of `subject` field (if text content present).
284+
- **Variable**: Value of `body` field (if text content present).
285+
- **Variable**: Value of `access_token` field (if text content present and token provided).
286+
- **Variable**: Value of `refresh_token` field (if text content present and token provided).
287+
- **Variable**: Value of `Image` (if image content present).
288+
289+
2. **Text format**: Binary-encoded fields with the following structure:
290+
291+
- **1 byte**: Content type indicator
292+
- **1 byte**: Length of `from` field (if text content present).
293+
- **2 bytes**: Length of `body` field (if text content present).
294+
- **2 bytes**: Length of `access_token` field (if text content present and optional).
295+
- **2 bytes**: Length of `refresh_token` field (if text content present and optional).
296+
- **2 bytes**: `Image Session ID` (if image content present).
297+
- **1 byte**: `Image Segment info` (if image content present).
298+
- **2 bytes**: Length of `Image` (if image content present).
299+
- **Variable**: Value of `from` field (if text content present).
300+
- **Variable**: Value of `body` field (if text content present).
301+
- **Variable**: Value of `access_token` field (if text content present and token provided).
302+
- **Variable**: Value of `refresh_token` field (if text content present and token provided).
303+
- **Variable**: Value of `Image` (if image content present).
304+
305+
3. **Message format**: Binary-encoded fields with the following structure:
306+
307+
- **1 byte**: Content type indicator
308+
- **1 byte**: Length of `from` field (if text content present).
309+
- **2 bytes**: Length of `to` field (if text content present).
310+
- **2 bytes**: Length of `body` field (if text content present).
311+
- **2 bytes**: `Image Session ID` (if image content present).
312+
- **1 byte**: `Image Segment info` (if image content present).
313+
- **2 bytes**: Length of `Image` (if image content present).
314+
- **Variable**: Value of `from` field (if text content present).
315+
- **Variable**: Value of `to` field (if text content present).
316+
- **Variable**: Value of `body` field (if text content present).
317+
- **Variable**: Value of `Image` (if image content present).
318+
319+
4. **Test format**: Binary-encoded fields with the following structure:
320+
321+
- **1 byte**: Length of `from` field (if text content present).
322+
- **Variable**: Value of `from` field (considered the test ID, if text content present).
323+
185324
## Supported Payload Versions
186325

187-
| **Version** | **Hexadecimal Value** | **Decimal Value** | **Description** |
188-
| ------------------------ | --------------------- | ----------------- | ----------------------------------------------------------- |
189-
| [v0](#payload-format-v0) | `None` | `None` | No explicit version marker, backward-compatible formats. |
190-
| [v1](#payload-format-v1) | `0x01` | `1` | Includes a version marker as the first byte of the payload. |
191-
| [v2](#payload-format-v1) | `0x02` | `2` | Includes a version marker as the first byte of the payload. |
326+
| **Version** | **Hexadecimal Value** | **Decimal Value** | **Description** |
327+
| ------------------------ | --------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
328+
| [v0](#payload-format-v0) | `None` | `None` | Legacy format with 4-byte ciphertext length and no version marker. Device ID has variable length without explicit length field. |
329+
| [v1](#payload-format-v1) | `0x01` | `1` | Introduces version marker, 2-byte ciphertext length, explicit device ID length field, and language code support. Uses Content Format V1 with 1-byte token length fields. |
330+
| [v2](#payload-format-v2) | `0x02` | `2` | Same payload structure as V1 but uses Content Format V2 with 2-byte token length fields for better support of longer access/refresh tokens. |
331+
| [v3](#payload-format-v3) | `0x03` | `3` | Same payload structure as V1/V2 but uses Content Format V3 with bitmap-based field presence indication for more efficient encoding. |
332+
| [v4](#payload-format-v4) | `0x04` | `4` | Same payload structure as V1/V2/V3 but uses Content Format V4 with support for image content. |
192333

193334
## Payload Format V0
194335

@@ -320,3 +461,99 @@ payload = (
320461
encoded = base64.b64encode(payload).decode("utf-8")
321462
print(encoded)
322463
```
464+
465+
## Payload Format V3
466+
467+
> [See available versions](#supported-payload-versions)
468+
469+
### Message Payload
470+
471+
- **Format**:
472+
- **1 byte**: Version Marker. [See available versions](#supported-payload-versions).
473+
- **2 bytes**: Ciphertext Length.
474+
- **1 byte**: Device ID Length.
475+
- **1 byte**: Platform shortcode.
476+
- **Variable**: Ciphertext. (encrypted [Content Format V3](#content-format-v3)).
477+
- **Variable**: Device ID.
478+
- **2 bytes**: Language Code (ISO 639-1 format).
479+
480+
> [!NOTE]
481+
>
482+
> For detailed instructions on using the Double Ratchet algorithm to create ciphertext, refer to the [smswithoutborders_lib_sig documentation](https://github.com/smswithoutborders/lib_signal_double_ratchet_python?tab=readme-ov-file#double-ratchet-implementations).
483+
484+
#### Visual Representation:
485+
486+
```plaintext
487+
+----------------+-------------------+------------------+--------------------+-----------------+-----------------+---------------+
488+
| Version Marker | Ciphertext Length | Device ID Length | Platform shortcode | Ciphertext | Device ID | Language Code |
489+
| (1 byte) | (2 bytes) | (1 byte) | (1 byte) | (Variable size) | (Variable size) | (2 bytes) |
490+
+----------------+-------------------+------------------+--------------------+-----------------+-----------------+---------------+
491+
```
492+
493+
```python
494+
version_marker = b'\x03'
495+
platform_shortcode = b'g'
496+
language_code = b'en'
497+
device_id = b'...'
498+
encrypted_content = b'...'
499+
500+
payload = (
501+
version_marker +
502+
struct.pack("<H", len(encrypted_content)) +
503+
bytes([len(device_id)]) +
504+
platform_shortcode +
505+
encrypted_content +
506+
device_id +
507+
language_code
508+
)
509+
encoded = base64.b64encode(payload).decode("utf-8")
510+
print(encoded)
511+
```
512+
513+
## Payload Format V4
514+
515+
> [See available versions](#supported-payload-versions)
516+
517+
### Message Payload
518+
519+
- **Format**:
520+
- **1 byte**: Version Marker. [See available versions](#supported-payload-versions).
521+
- **2 bytes**: Ciphertext Length.
522+
- **1 byte**: Device ID Length.
523+
- **1 byte**: Platform shortcode.
524+
- **Variable**: Ciphertext. (encrypted [Content Format V4](#content-format-v4)).
525+
- **Variable**: Device ID.
526+
- **2 bytes**: Language Code (ISO 639-1 format).
527+
528+
> [!NOTE]
529+
>
530+
> For detailed instructions on using the Double Ratchet algorithm to create ciphertext, refer to the [smswithoutborders_lib_sig documentation](https://github.com/smswithoutborders/lib_signal_double_ratchet_python?tab=readme-ov-file#double-ratchet-implementations).
531+
532+
#### Visual Representation:
533+
534+
```plaintext
535+
+----------------+-------------------+------------------+--------------------+-----------------+-----------------+---------------+
536+
| Version Marker | Ciphertext Length | Device ID Length | Platform shortcode | Ciphertext | Device ID | Language Code |
537+
| (1 byte) | (2 bytes) | (1 byte) | (1 byte) | (Variable size) | (Variable size) | (2 bytes) |
538+
+----------------+-------------------+------------------+--------------------+-----------------+-----------------+---------------+
539+
```
540+
541+
```python
542+
version_marker = b'\x04'
543+
platform_shortcode = b'g'
544+
language_code = b'en'
545+
device_id = b'...'
546+
encrypted_content = b'...'
547+
548+
payload = (
549+
version_marker +
550+
struct.pack("<H", len(encrypted_content)) +
551+
bytes([len(device_id)]) +
552+
platform_shortcode +
553+
encrypted_content +
554+
device_id +
555+
language_code
556+
)
557+
encoded = base64.b64encode(payload).decode("utf-8")
558+
print(encoded)
559+
```

‎requirements.txt‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
1-
fastapi[standard]==0.117.1
1+
fastapi[standard]==0.118.0
22
GitPython==3.1.45
33
grpc-interceptor==0.15.4
4-
grpcio==1.75.0
5-
grpcio-testing==1.75.0
6-
grpcio-tools==1.75.0
4+
grpcio==1.75.1
5+
grpcio-testing==1.75.1
6+
grpcio-tools==1.75.1
77
peewee>=3.17.8
8-
phonenumbers==9.0.14
8+
phonenumbers==9.0.15
99
pymysql==1.1.2
1010
requests==2.32.5
11-
sentry-sdk[grpcio]==2.38.0
11+
sentry-sdk[grpcio]==2.40.0
1212
tqdm==4.67.1
13-
twilio==9.8.1
13+
twilio==9.8.3

0 commit comments

Comments
 (0)