Consensus header and state serialization - use protobufs for serialization instead of other methods (like asn1)

[HagarMeir]

The current approach while functional, has several limitations that impact maintainability, performance, and ecosystem compatibility.

Why Protocol Buffers?

1. Industry Standard & Ecosystem Compatibility

• Protocol Buffers is the de facto standard for structured data serialization in modern distributed systems
• Already used extensively throughout the Fabric ecosystem (common protos, block structures, etc.)
• Better tooling support across multiple languages and platforms
• Active development and community support from Google

2. Performance Benefits

• Faster serialization/deserialization: Protobuf is optimized for speed with generated code
• Smaller wire format: More compact binary representation compared to ASN.1
• Zero-copy operations: Protobuf supports efficient memory handling
• Benchmarks show 2-3x faster encoding/decoding in typical scenarios

3. Schema Evolution & Backward Compatibility

• Field numbering: Protobuf's field numbering system makes schema evolution explicit and safe
• Optional fields: Clear semantics for optional vs required fields
• Deprecation support: Can mark fields as deprecated without breaking compatibility
• Version management: Better support for gradual migration and rollback

4. Type Safety & Validation

• Strongly typed: Generated code provides compile-time type safety
• Validation: Built-in validation for required fields and data types
• Default values: Clear semantics for default values
• Enum support: Better enum handling with forward/backward compatibility

5. Developer Experience

• Code generation: Automatic generation of serialization code from .proto files
• IDE support: Better autocomplete, navigation, and refactoring support
• Documentation: Proto files serve as self-documenting schema definitions
• Debugging: Better tooling for inspecting and debugging serialized data

6. Consistency with Fabric Architecture

• Aligns with Hyperledger Fabric's existing use of Protocol Buffers
• Reduces cognitive load by using a single serialization format
• Easier integration with other Fabric components
• Consistent patterns across the codebase

Problems with ASN.1

• Limited tooling: Fewer tools for debugging, validation, and schema management
• Complexity: ASN.1 syntax and rules are more complex and error-prone
• Maintenance burden: Less common in modern Go codebases, harder to find expertise
• Performance: Generally slower than Protobuf for typical use cases
• Ecosystem mismatch: Not the standard in the Fabric ecosystem