Skip to content

Commit ed73a87

Browse files
committed
feat: Add v1.2.0 feature documentation for binary format, versioning, adaptive encoding, and developer tools, and update README badges, media type, and test statistics.
1 parent 023d515 commit ed73a87

2 files changed

Lines changed: 266 additions & 18 deletions

File tree

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# Zero Overhead Notation (ZON) Format
22

33
[![GitHub stars](https://img.shields.io/github/stars/ZON-Format/ZON?style=social&label=Star)](https://github.com/ZON-Format/ZON)
4-
[![PyPI downloads](https://img.shields.io/pypi/dm/zon-format?color=red)](https://pypi.org/project/zon-format/)
4+
[![Downloads](https://static.pepy.tech/badge/zon-format/month)](https://pepy.tech/project/zon-format)
55
[![PyPI version](https://img.shields.io/pypi/v/zon-format.svg)](https://pypi.org/project/zon-format/)
66
[![Python](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
77
[![Tests](https://img.shields.io/badge/tests-340%2F340%20passing-brightgreen.svg)](#quality--testing)

zon-format/README.md

Lines changed: 265 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,18 @@
11
# Zero Overhead Notation (ZON) Format
22

33
[![GitHub stars](https://img.shields.io/github/stars/ZON-Format/ZON?style=social&label=Star)](https://github.com/ZON-Format/ZON)
4-
[![PyPI downloads](https://img.shields.io/pypi/dm/zon-format?color=red)](https://pypi.org/project/zon-format/)
4+
[![Downloads](https://static.pepy.tech/badge/zon-format/month)](https://pepy.tech/project/zon-format)
55
[![PyPI version](https://img.shields.io/pypi/v/zon-format.svg)](https://pypi.org/project/zon-format/)
66
[![Python](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
7-
[![Tests](https://img.shields.io/badge/tests-121%2F121%20passing-brightgreen.svg)](#quality--testing)
8-
[![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
7+
[![Tests](https://img.shields.io/badge/tests-340%2F340%20passing-brightgreen.svg)](#quality--testing)
8+
![CodeRabbit Pull Request Reviews](https://img.shields.io/coderabbit/prs/github/ZON-Format/ZON?utm_source=oss&utm_medium=github&utm_campaign=ZON-Format%2FZON&labelColor=171717&color=FF570A&link=https%3A%2F%2Fcoderabbit.ai&label=CodeRabbit+Reviews)
9+
[![License](https://img.shields.io/badge/license-MIT-green.svg)](../LICENSE)
910

10-
# ZON → JSON is dead. TOON was cute. ZON just won. (Now in Python v1.1.0)
11+
# ZON → JSON is dead. TOON was cute. ZON just won. (Python v1.2.0 - Now with Binary Format, Versioning & Enterprise Tools)
1112

1213
**Zero Overhead Notation** - A compact, human-readable way to encode JSON for LLMs.
1314

14-
**File Extension:** `.zonf` | **Media Type:** `text/zon` | **Encoding:** UTF-8
15+
**File Extension:** `.zonf` | **Media Type:** `text/zonf` | **Encoding:** UTF-8
1516

1617
ZON is a token-efficient serialization format designed for LLM workflows. It achieves 35-50% token reduction vs JSON through tabular encoding, single-character primitives, and intelligent compression (Delta, Dictionary) while maintaining 100% data fidelity.
1718

@@ -24,6 +25,12 @@ Think of it like CSV for complex data - keeps the efficiency of tables where it
2425

2526
```bash
2627
pip install zon-format
28+
29+
# Install with UV (5-10x faster than pip)
30+
uv pip install zon-format
31+
32+
# Or for UV-based projects
33+
uv add zon-format
2734
```
2835

2936
> [!TIP]
@@ -419,12 +426,162 @@ ZON is **immune to code injection attacks** that plague other formats:
419426

420427
---
421428

429+
## New in v1.2.0: Enterprise Features
430+
431+
### Binary Format (ZON-B)
432+
433+
Compact binary encoding with 40-60% space savings vs JSON:
434+
435+
```python
436+
from zon import encode_binary, decode_binary
437+
438+
# Encode to binary
439+
data = {"users": [{"id": 1, "name": "Alice"}, {"id": 2, "name": "Bob"}]}
440+
binary = encode_binary(data) # 40-60% smaller than JSON
441+
442+
# Decode from binary
443+
decoded = decode_binary(binary)
444+
```
445+
446+
**Features:**
447+
- MessagePack-inspired format with magic header (`ZNB\x01`)
448+
- Full type support for all ZON primitives
449+
- Perfect round-trip fidelity
450+
- Ideal for storage, APIs, and network transmission
451+
452+
### Versioning & Migration System
453+
454+
Document-level schema versioning with automatic migrations:
455+
456+
```python
457+
from zon import embed_version, extract_version, ZonMigrationManager
458+
459+
# Embed version metadata
460+
versioned = embed_version(data, "2.0.0", "user-schema")
461+
462+
# Extract version info
463+
meta = extract_version(versioned)
464+
465+
# Setup migration manager
466+
manager = ZonMigrationManager()
467+
manager.register_migration("1.0.0", "2.0.0", upgrade_function)
468+
469+
# Automatically migrate
470+
migrated = manager.migrate(old_data, "1.0.0", "2.0.0")
471+
```
472+
473+
**Features:**
474+
- Semantic versioning support
475+
- BFS-based migration path finding
476+
- Backward/forward compatibility checking
477+
- Chained migrations for complex upgrades
478+
479+
### Adaptive Encoding
480+
481+
Three encoding modes optimized for different use cases:
482+
483+
```python
484+
from zon import encode_adaptive, recommend_mode, AdaptiveEncodeOptions
485+
486+
# Auto-recommend best mode
487+
recommendation = recommend_mode(data)
488+
# {'mode': 'compact', 'confidence': 0.95, 'reason': 'Large uniform array...'}
489+
490+
# Compact mode - maximum compression
491+
compact = encode_adaptive(data, AdaptiveEncodeOptions(mode='compact'))
492+
493+
# Readable mode - pretty-printed with indentation
494+
readable = encode_adaptive(data, AdaptiveEncodeOptions(mode='readable', indent=2))
495+
496+
# LLM-optimized - balanced for AI workflows
497+
llm = encode_adaptive(data, AdaptiveEncodeOptions(mode='llm-optimized'))
498+
```
499+
500+
**Encoding Modes:**
501+
502+
| Mode | Best For | Features |
503+
|------|----------|----------|
504+
| **compact** | Production APIs | Maximum compression, T/F booleans |
505+
| **readable** | Config files | Multi-line indentation, human-friendly |
506+
| **llm-optimized** | AI workflows | true/false booleans, no type coercion |
507+
508+
**Readable Mode Example:**
509+
```zon
510+
metadata:{
511+
generated:2025-01-01T12:00:00Z
512+
version:1.2.0
513+
}
514+
515+
users:@(2):id,name,role
516+
1,Alice,admin
517+
2,Bob,user
518+
```
519+
520+
### Developer Tools
521+
522+
Comprehensive utilities for working with ZON data:
523+
524+
```python
525+
from zon import size, compare_formats, analyze, ZonValidator
526+
527+
# Analyze data size across formats
528+
comparison = compare_formats(data)
529+
# {'json': {'size': 1200, 'percentage': 100.0},
530+
# 'zon': {'size': 800, 'percentage': 66.7},
531+
# 'binary': {'size': 480, 'percentage': 40.0}}
532+
533+
# Data complexity analysis
534+
analysis = analyze(data)
535+
# {'depth': 3, 'complexity': 'moderate', 'recommended_format': 'zon'}
536+
537+
# Enhanced validation
538+
validator = ZonValidator()
539+
result = validator.validate(zon_string)
540+
if not result.is_valid:
541+
for error in result.errors:
542+
print(f"Error at line {error.line}: {error.message}")
543+
```
544+
545+
**Tools Available:**
546+
- `size()` - Calculate data size in different formats
547+
- `compare_formats()` - Compare JSON/ZON/Binary sizes
548+
- `analyze()` - Comprehensive data structure analysis
549+
- `infer_schema()` - Automatic schema inference
550+
- `ZonValidator` - Enhanced validation with linting rules
551+
- `expand_print()` - Pretty-printer for readable formatting
552+
553+
### Complete API
554+
555+
```python
556+
from zon import (
557+
# Core encoding
558+
encode, decode, encode_llm,
559+
560+
# Adaptive encoding (v1.2.0)
561+
encode_adaptive, recommend_mode, AdaptiveEncodeOptions,
562+
563+
# Binary format (v1.2.0)
564+
encode_binary, decode_binary,
565+
566+
# Versioning (v1.2.0)
567+
embed_version, extract_version, compare_versions,
568+
is_compatible, strip_version, ZonMigrationManager,
569+
570+
# Developer tools (v1.2.0)
571+
size, compare_formats, analyze, infer_schema,
572+
compare, is_safe, ZonValidator, expand_print
573+
)
574+
```
575+
576+
---
577+
422578
## Quality & Security
423579

424580
### Data Integrity
425-
- **Unit tests:** 94/94 passed (+66 new validation/security/conformance tests)
426-
- **Roundtrip tests:** 27/27 datasets verified
581+
- **Unit tests:** 340/340 passed (v1.2.0 adds 103 new tests for binary, versioning, tools)
582+
- **Roundtrip tests:** 27/27 datasets verified + 51 cross-language examples
427583
- **No data loss or corruption**
584+
- **Cross-language compatibility:** 51% exact match with TypeScript v1.3.0
428585

429586
### Security Limits (DOS Prevention)
430587

@@ -565,6 +722,56 @@ logs:"[{id:101,level:INFO},{id:102,level:WARN}]"
565722

566723
---
567724

725+
## Encoding Modes (New in v1.2.0)
726+
727+
ZON now provides **three encoding modes** optimized for different use cases:
728+
729+
### Mode Overview
730+
731+
| Mode | Best For | Token Efficiency | Human Readable | LLM Clarity | Default |
732+
|------|----------|------------------|----------------|-------------|---------|
733+
| **compact** | Production APIs, LLMs | ⭐⭐⭐⭐⭐ | ⭐⭐ | ⭐⭐⭐⭐ | ✅ YES |
734+
| **llm-optimized** | AI workflows | ⭐⭐⭐⭐ | ⭐⭐⭐ | ⭐⭐⭐⭐⭐ | |
735+
| **readable** | Config files, debugging | ⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐ | |
736+
737+
### Adaptive Encoding
738+
739+
```python
740+
from zon import encode_adaptive, AdaptiveEncodeOptions, recommend_mode
741+
742+
# Use compact mode (default - maximum compression)
743+
output = encode_adaptive(data)
744+
745+
# Use readable mode (human-friendly)
746+
output = encode_adaptive(data, AdaptiveEncodeOptions(mode='readable'))
747+
748+
# Use LLM-optimized mode (balanced for AI)
749+
output = encode_adaptive(data, AdaptiveEncodeOptions(mode='llm-optimized'))
750+
751+
# Get recommendation for your data
752+
recommendation = recommend_mode(data)
753+
print(f"Use {recommendation['mode']} mode: {recommendation['reason']}")
754+
```
755+
756+
### Mode Details
757+
758+
**Compact Mode (Default)**
759+
- Maximum compression using tables and abbreviations (`T`/`F` for booleans)
760+
- Dictionary compression for repeated values
761+
- Best for production APIs and cost-sensitive LLM workflows
762+
763+
**LLM-Optimized Mode**
764+
- Balances token efficiency with AI comprehension
765+
- Uses `true`/`false` instead of `T`/`F` for better LLM understanding
766+
- Disables dictionary compression for clarity
767+
768+
**Readable Mode**
769+
- Human-friendly formatting with proper indentation
770+
- Perfect for configuration files and debugging
771+
- Easy editing and version control
772+
773+
---
774+
568775
## API Reference
569776

570777
### `zon.encode(data: Any) -> str`
@@ -584,6 +791,47 @@ zon_str = zon.encode({
584791

585792
**Returns:** ZON-formatted string
586793

794+
### `zon.encode_adaptive(data: Any, options: AdaptiveEncodeOptions = None) -> str`
795+
796+
Encodes Python data using adaptive mode selection (New in v1.2.0).
797+
798+
```python
799+
from zon import encode_adaptive, AdaptiveEncodeOptions
800+
801+
# Compact mode (default)
802+
output = encode_adaptive(data)
803+
804+
# Readable mode with custom indentation
805+
output = encode_adaptive(
806+
data,
807+
AdaptiveEncodeOptions(mode='readable', indent=4)
808+
)
809+
810+
# With debug information
811+
result = encode_adaptive(
812+
data,
813+
AdaptiveEncodeOptions(mode='compact', debug=True)
814+
)
815+
print(result.decisions) # See encoding decisions
816+
```
817+
818+
**Returns:** ZON-formatted string or `AdaptiveEncodeResult` if debug=True
819+
820+
### `zon.recommend_mode(data: Any) -> dict`
821+
822+
Analyzes data and recommends optimal encoding mode (New in v1.2.0).
823+
824+
```python
825+
from zon import recommend_mode
826+
827+
recommendation = recommend_mode(my_data)
828+
print(f"Use {recommendation['mode']} mode")
829+
print(f"Confidence: {recommendation['confidence']}")
830+
print(f"Reason: {recommendation['reason']}")
831+
```
832+
833+
**Returns:** Dictionary with mode, confidence, reason, and metrics
834+
587835
### `zon.decode(zon_string: str, strict: bool = True) -> Any`
588836

589837
Decodes ZON format back to Python data.
@@ -744,9 +992,9 @@ zon_products = zon.encode(products)
744992

745993
## Documentation
746994

747-
Comprehensive guides and references are available in the [`zon-format/docs/`](./zon-format/docs/) directory:
995+
Comprehensive guides and references are available in the [`./docs/`](./docs/) directory:
748996

749-
### 📖 [Syntax Cheatsheet](./zon-format/docs/syntax-cheatsheet.md)
997+
### 📖 [Syntax Cheatsheet](./docs/syntax-cheatsheet.md)
750998
Quick reference for ZON format syntax with practical examples.
751999

7521000
**What's inside:**
@@ -761,15 +1009,15 @@ Quick reference for ZON format syntax with practical examples.
7611009

7621010
---
7631011

764-
### 🔧 [API Reference](./zon-format/docs/api-reference.md)
1012+
### 🔧 [API Reference](./docs/api-reference.md)
7651013
Complete API documentation for `zon-format` v1.0.4.
7661014

7671015
**What's inside:**
7681016
- `encode()` function - detailed parameters and examples
7691017
- `decode()` function - detailed parameters and examples
7701018
- Python type definitions
7711019

772-
### 📘 [Complete Specification](./SPEC.md)
1020+
### 📘 [Complete Specification](../SPEC.md)
7731021

7741022
Comprehensive formal specification including:
7751023
- Data model and encoding rules
@@ -781,16 +1029,16 @@ Comprehensive formal specification including:
7811029

7821030
### 📚 Other Documentation
7831031

784-
- **[API Reference](./zon-format/docs/api-reference.md)** - Encoder/decoder API, options, error codes
785-
- **[Syntax Cheatsheet](./zon-format/docs/syntax-cheatsheet.md)** - Quick reference guide
786-
- **[LLM Best Practices](./zon-format/docs/llm-best-practices.md)** - Using ZON with LLMs
1032+
- **[API Reference](./docs/api-reference.md)** - Encoder/decoder API, options, error codes
1033+
- **[Syntax Cheatsheet](./docs/syntax-cheatsheet.md)** - Quick reference guide
1034+
- **[LLM Best Practices](./docs/llm-best-practices.md)** - Using ZON with LLMs
7871035

7881036
---
7891037

7901038
## Links
7911039

7921040
- [PyPI Package](https://pypi.org/project/zon-format/)
793-
- [Changelog](./zon-format/CHANGELOG.md)
1041+
- [Changelog](./CHANGELOG.md)
7941042
- [GitHub Repository](https://github.com/ZON-Format/ZON)
7951043
- [GitHub Issues](https://github.com/ZON-Format/ZON/issues)
7961044
- [TypeScript Implementation](https://github.com/ZON-Format/zon-TS)
@@ -811,10 +1059,10 @@ Contributions welcome! Please:
8111059

8121060
Copyright (c) 2025 ZON-FORMAT (Roni Bhakta)
8131061

814-
MIT License - see [LICENSE](LICENSE) for details.
1062+
MIT License - see [LICENSE](../LICENSE) for details.
8151063

8161064
---
8171065

8181066
**Made with ❤️ for the LLM community**
8191067

820-
*ZON v1.0.4 - Token efficiency that scales with complexity*
1068+
*ZON v1.2.0 - Token efficiency that scales with complexity, now with adaptive encoding*

0 commit comments

Comments
 (0)