Data Encoding & Transformation

Data Encoding & Transformation

Table of Contents

1. Introduction
2. CBOR Encoding/Decoding Implementation
3. Base64URL Encoding Conventions
4. WebAuthn JSON Browser Ponyfill
5. Cryptographic Value Encoding
6. Server-Side Encoding Pipeline
7. Security Considerations
8. Complete Transformation Flow
9. Testing and Validation
10. Troubleshooting Guide

Introduction

The Post-Quantum WebAuthn Platform implements a sophisticated data encoding and transformation system that handles the conversion between various data formats required for secure authentication. This system ensures interoperability between client and server components while maintaining security through proper encoding practices.

The platform supports multiple encoding formats including CBOR (Concise Binary Object Representation), base64url encoding, and JSON transformations. These encoding mechanisms are essential for handling cryptographic values, WebAuthn API objects, and CTAP2 protocol messages.

CBOR Encoding/Decoding Implementation

Core CBOR Functions

The CBOR implementation in fido2/cbor.py provides minimal yet comprehensive support for FIDO 2 CTAP protocols. The implementation focuses on essential types and follows strict canonical encoding rules.

classDiagram
class CBORImplementation {
+encode(data) bytes
+decode(data) CborType
+decode_from(data) Tuple[Any, bytes]
-dump_int(data, mt) bytes
-dump_bool(data) bytes
-dump_list(data) bytes
-dump_dict(data) bytes
-dump_bytes(data) bytes
-dump_text(data) bytes
-load_int(ai, data) Tuple[int, bytes]
-load_bool(ai, data) Tuple[bool, bytes]
-load_bytes(ai, data) Tuple[bytes, bytes]
-load_text(ai, data) Tuple[str, bytes]
-load_array(ai, data) Tuple[Sequence, bytes]
-load_map(ai, data) Tuple[Mapping, bytes]
}
class SerializationHandlers {
+_SERIALIZERS : Sequence[Tuple[Type, Callable]]
+_DESERIALIZERS : Dict[int, Callable]
+_sort_keys(entry) Tuple
}
CBORImplementation --> SerializationHandlers : uses

Loading

Diagram sources

• fido2/cbor.py

Integer Encoding Algorithm

The CBOR integer encoding uses a flexible approach based on value magnitude:

Value Range	Encoding Method	Byte Pattern
0-23	Direct encoding	Single byte: value
24-255	One-byte length	0x18 + 1-byte value
256-65535	Two-byte length	0x19 + 2-byte value
65536-4294967295	Four-byte length	0x1A + 4-byte value
4294967296+	Eight-byte length	0x1B + 8-byte value

Canonical Key Ordering

The implementation enforces canonical CBOR key ordering for dictionaries, ensuring deterministic serialization:

flowchart TD
A["Dictionary Input"] --> B["Extract Keys"]
B --> C["Sort Keys by Type"]
C --> D["Sort by Length"]
D --> E["Sort Lexicographically"]
E --> F["Encode Sorted Pairs"]
F --> G["Final CBOR Output"]
C --> C1["Integer Keys"]
C --> C2["String Keys"]
C --> C3["Byte String Keys"]
C1 --> C1A["Sort by Numeric Value"]
C2 --> C2A["Sort by ASCII Order"]
C3 --> C3A["Sort by Byte Comparison"]

Loading

Diagram sources

• fido2/cbor.py

Section sources

• fido2/cbor.py

Base64URL Encoding Conventions

Utility Functions

The fido2/utils.py module provides robust base64url encoding functions that handle binary data conversion for JSON transmission.

classDiagram
class Base64Utils {
+websafe_encode(data : bytes) str
+websafe_decode(data : Union[str, bytes]) bytes
+sha256(data : bytes) bytes
+hmac_sha256(key : bytes, data : bytes) bytes
+bytes2int(value : bytes) int
+int2bytes(value : int, minlen : int) bytes
}
class ByteBuffer {
+unpack(fmt : str) Any
+read(size : Optional[int]) bytes
}
Base64Utils --> ByteBuffer : uses

Loading

Diagram sources

• fido2/utils.py

Encoding Process

The base64url encoding follows RFC 4648 standards with URL-safe character substitution:

Standard Base64	URL-Safe Base64	Purpose
+	-	Plus sign replacement
/	_	Forward slash replacement
=	(removed)	Padding removal

Binary Data Conversion

The utility functions handle bidirectional conversion between binary data and string representations:

sequenceDiagram
participant Client as "Client Application"
participant Utils as "Base64 Utils"
participant Server as "Server"
Client->>Utils : bytes2int(binary_data)
Utils-->>Client : integer_value
Client->>Utils : int2bytes(int_value, minlen)
Utils-->>Client : padded_binary_data
Client->>Utils : websafe_encode(binary_data)
Utils-->>Client : base64url_string
Client->>Server : JSON with base64url_string
Server->>Utils : websafe_decode(base64url_string)
Utils-->>Server : original_binary_data

Loading

Diagram sources

• fido2/utils.py

Section sources

• fido2/utils.py

WebAuthn JSON Browser Ponyfill

ArrayBuffer to JSON Conversion

The webauthn-json.browser-ponyfill.js provides seamless conversion between browser WebAuthn API objects and JSON representations.

classDiagram
class WebAuthnPonyfill {
+create(options) Promise~PublicKeyCredential~
+get(options) Promise~PublicKeyCredential~
+createRequestFromJSON(requestJSON) CredentialCreationOptions
+getRequestFromJSON(requestJSON) CredentialRequestOptions
+supported() boolean
}
class Converter {
+base64urlToBuffer(base64urlString) ArrayBuffer
+bufferToBase64url(buffer) string
+convert(conversionFn, schema, input) Any
}
class SchemaDefinitions {
+credentialCreationOptions : Object
+credentialRequestOptions : Object
+publicKeyCredentialWithAttestation : Object
+publicKeyCredentialWithAssertion : Object
}
WebAuthnPonyfill --> Converter : uses
WebAuthnPonyfill --> SchemaDefinitions : defines

Loading

Diagram sources

• server/static/scripts/shared/webauthn-json.browser-ponyfill.js

Recursive Credential Transformation

The conversion process handles nested credential objects with recursive transformation:

flowchart TD
A["Browser WebAuthn API"] --> B["PublicKeyCredential"]
B --> C["Raw ArrayBuffer"]
C --> D["base64urlToBuffer()"]
D --> E["JSON String"]
E --> F["JSON.parse()"]
F --> G["Credential Object"]
G --> H["Recursive Schema Validation"]
H --> I["Nested Object Conversion"]
I --> J["Final JSON Output"]
K["Server JSON"] --> L["JSON.stringify()"]
L --> M["base64url Encoding"]
M --> N["Credential Creation"]
N --> O["PublicKeyCredential"]
O --> P["ArrayBuffer Response"]

Loading

Diagram sources

• server/static/scripts/shared/webauthn-json.browser-ponyfill.js

Schema Definitions

The ponyfill defines comprehensive schemas for credential objects:

Schema Type	Purpose	Key Fields
credentialCreationOptions	Registration requests	rp, user, challenge, pubKeyCredParams
credentialRequestOptions	Authentication requests	challenge, allowCredentials, userVerification
publicKeyCredentialWithAttestation	Registration responses	attestationObject, clientDataJSON
publicKeyCredentialWithAssertion	Authentication responses	authenticatorData, signature, clientDataJSON

Section sources

• server/static/scripts/shared/webauthn-json.browser-ponyfill.js

Cryptographic Value Encoding

Public Key Encoding

The platform supports multiple public key formats with automatic detection and conversion:

classDiagram
class PublicKeyEncoding {
+_coerce_mldsa_public_key_bytes(value, parameter_set) bytes
+_unwrap_mldsa_subject_public_key(payload, parameter_set) Tuple
+_extract_subject_public_key_from_spki(spki_der) bytes
+_parse_spki_algorithm_info(spki_der) Tuple
}
class COSEKey {
+ALGORITHM : int
+verify(message, signature) None
+from_cryptography_key(public_key) CoseKey
+set_assertion_debug_data(authenticator_data, client_data_json) None
}
PublicKeyEncoding --> COSEKey : creates

Loading

Diagram sources

• fido2/cose.py

Signature Encoding

Digital signatures undergo canonical encoding to ensure consistency:

sequenceDiagram
participant App as "Application"
participant Crypto as "Cryptography Library"
participant COSE as "COSE Encoder"
participant Network as "Network Transport"
App->>Crypto : Sign(message, private_key)
Crypto-->>App : DER Signature
App->>COSE : Canonicalize Signature
COSE->>COSE : Validate DER Format
COSE->>COSE : Apply Low-S Form
COSE-->>App : Canonical Signature
App->>Network : Send CBOR Encoded Signature
Network-->>App : Confirmation

Loading

Diagram sources

• fido2/cose.py

Authenticator Data Encoding

Authenticator data combines multiple components into a structured format:

Component	Size	Purpose
RpId Hash	32 bytes	Relying Party identification
Flags	1 byte	Authenticator state flags
Sign Count	4 bytes	Counter for signatures
Attested Credential Data	Variable	Credential information
Extensions	Variable	Additional data

Section sources

• fido2/cose.py

Server-Side Encoding Pipeline

Multi-Format Encoder

The server-side encoder supports multiple output formats with automatic detection:

classDiagram
class EncoderPipeline {
+encode_payload_text(value, target_format) Dict
+_encode_json_value(parsed) Dict
+_encode_cbor_value(parsed) Dict
+_encode_client_data(parsed) Dict
+_encode_authenticator_data(parsed) Dict
+_encode_attestation_object(parsed) Dict
+_encode_x509_certificate(parsed) Dict
}
class FormatHandlers {
+_ENCODING_HANDLERS : Dict[str, Callable]
+_normalize_encoding_format(value) str
+_prepare_encoder_response(base_type, data) Dict
}
class CanonicalEncoder {
+_canonical_cbor_dumps(value) bytes
+_canonicalize_cbor_structure(value) Any
+_encode_map(value) bytes
+_encode_array(value) bytes
}
EncoderPipeline --> FormatHandlers : uses
EncoderPipeline --> CanonicalEncoder : uses

Loading

Diagram sources

• server/server/decoder/encode.py

Binary Data Processing

The decoder handles various binary input formats with intelligent detection:

flowchart TD
A["Input Text"] --> B{"Format Detection"}
B --> |Hex| C["Hex Decoder"]
B --> |Base64| D["Base64 Decoder"]
B --> |Base64URL| E["Base64URL Decoder"]
B --> |JSON| F["JSON Parser"]
C --> G["Binary Data"]
D --> G
E --> G
F --> H["Structured Data"]
G --> I["Format-Specific Decoding"]
H --> I
I --> J["Output Structure"]

Loading

Diagram sources

• server/server/decoder/decode.py

Section sources

• server/server/decoder/encode.py
• server/server/decoder/decode.py

Security Considerations

Data Integrity Protection

The encoding system implements several security measures to prevent data corruption and ensure integrity:

Security Measure	Implementation	Purpose
Canonical CBOR	Strict ordering rules	Prevents replay attacks
Base64URL Validation	Character set enforcement	Prevents injection attacks
Length Limits	Input size validation	Prevents DoS attacks
Type Checking	Schema validation	Ensures data correctness

Interoperability Standards

The platform adheres to established standards for cross-platform compatibility:

• RFC 8949: CBOR encoding specification
• RFC 4648: Base64 encoding standards
• WebAuthn Specification: Browser API compatibility
• CTAP2 Protocol: Authenticator communication

Encoding Best Practices

1. Consistent Encoding: Always use canonical CBOR for cryptographic data
2. Proper Padding: Ensure base64url strings are properly padded
3. Type Safety: Validate data types before encoding
4. Error Handling: Implement comprehensive error checking
5. Format Detection: Automatically detect input formats

Complete Transformation Flow

Registration Flow Example

sequenceDiagram
participant User as "User"
participant Browser as "Browser"
participant Server as "Server"
participant Authenticator as "Authenticator"
User->>Browser : Initiate Registration
Browser->>Server : GET /register/options
Server-->>Browser : Registration Options (JSON)
Browser->>Browser : Convert to ArrayBuffer
Browser->>Authenticator : navigator.credentials.create()
Authenticator-->>Browser : PublicKeyCredential
Browser->>Browser : Convert to JSON
Browser->>Server : POST /register
Server->>Server : Decode CBOR
Server->>Server : Verify Attestation
Server-->>Browser : Registration Success

Loading

Authentication Flow Example

sequenceDiagram
participant User as "User"
participant Browser as "Browser"
participant Server as "Server"
participant Authenticator as "Authenticator"
User->>Browser : Initiate Authentication
Browser->>Server : GET /login/options
Server-->>Browser : Authentication Options (JSON)
Browser->>Browser : Convert to ArrayBuffer
Browser->>Authenticator : navigator.credentials.get()
Authenticator-->>Browser : PublicKeyCredential
Browser->>Browser : Convert to JSON
Browser->>Server : POST /login
Server->>Server : Decode CBOR
Server->>Server : Verify Signature
Server-->>Browser : Authentication Success

Loading

Testing and Validation

CBOR Test Vectors

The platform includes comprehensive test coverage for CBOR encoding:

Test Category	Coverage	Purpose
Integer Encoding	Positive/negative values	Validates encoding ranges
Key Ordering	Dictionary sorting	Ensures canonical output
Nested Structures	Arrays/maps	Tests recursion handling
Edge Cases	Boundary values	Validates robustness

Base64URL Test Cases

The utility functions include extensive testing for encoding/decoding:

flowchart TD
A["Test Input"] --> B["Encode"]
B --> C["Decode"]
C --> D["Compare"]
D --> E{"Match?"}
E --> |Yes| F["Pass"]
E --> |No| G["Fail"]
H["Edge Cases"] --> I["Empty Strings"]
H --> J["Special Characters"]
H --> K["Large Data"]
H --> L["Invalid Input"]
I --> A
J --> A
K --> A
L --> A

Loading

Section sources

• tests/test_cbor.py
• tests/test_utils.py

Troubleshooting Guide

Common Encoding Issues

Problem	Symptoms	Solution
CBOR Decoding Failure	Unexpected end of data	Check for truncated input
Base64URL Mismatch	Invalid characters	Verify padding and charset
Key Ordering Issues	Non-canonical output	Enable canonical mode
Type Conversion Errors	Schema validation failures	Validate input types

Debugging Techniques

1. Enable Logging: Use verbose logging for detailed transformation steps
2. Format Validation: Verify input format before processing
3. Step-by-Step Tracing: Break down complex transformations
4. Test Vector Comparison: Compare against known good outputs
5. Schema Validation: Ensure data conforms to expected structures

Performance Optimization

• Batch Processing: Group related operations for efficiency
• Caching: Store frequently used encodings
• Streaming: Process large data in chunks
• Memory Management: Release resources promptly