Skip to content

Commit 6f04f68

Browse files
author
test
committed
Merge branch 'main' into feat/tcp-connecotr-hardening
2 parents 337d202 + 275fbe4 commit 6f04f68

20 files changed

Lines changed: 896 additions & 37 deletions

File tree

CHANGELOG.md

Lines changed: 28 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

88
> **Note**: This is the global changelog for the AimDB project. For detailed changes to individual crates, see their respective CHANGELOG.md files:
9+
>
910
> - [aimdb-core/CHANGELOG.md](aimdb-core/CHANGELOG.md)
1011
> - [aimdb-codegen/CHANGELOG.md](aimdb-codegen/CHANGELOG.md)
1112
> - [aimdb-data-contracts/CHANGELOG.md](aimdb-data-contracts/CHANGELOG.md)
@@ -106,6 +107,10 @@ multi-step migration round-trip suite and a trybuild compile-fail harness.
106107

107108
### Added
108109

110+
- **Zero-allocation generated Postcard encoding (Issue #177).** `Linkable` gains a source-compatible `encode_into(&mut [u8])` fast path and generated Postcard records implement it with `postcard::to_slice`. Core reuses onebounded 256-byte scratch allocation per generated Postcard outbound route (raw builders choose their capacity) and falls back to the existing owned serializer when a value does not fit. The codec seam is gated by a 10,000-iteration allocation-counting bench (0 allocations, 0 bytes) plus
111+
host, `no_std` Cortex-M, and codegen-drift tests. The claim intentionally excludes connector-internal payload copies and the existing boxed connector reader future. ([aimdb-core](aimdb-core/CHANGELOG.md),
112+
[aimdb-data-contracts](aimdb-data-contracts/CHANGELOG.md),
113+
[aimdb-codegen](aimdb-codegen/CHANGELOG.md))
109114
- **Embassy MQTT client TLS — `mqtts://` for embedded, WP7 ([design 044](docs/design/044-embassy-mqtt-tls.md)).** The Embassy MQTT connector can now dial a broker over TLS 1.3 (`embedded-tls`, pure-Rust `rustpki` certificate verification with `rsa`/`p384` so public CA chains verify out of the box), with SNI/hostname verification, DNS resolution, and a connector-internal SNTP time source that gates the first handshake so certificate validity is always checked against real time. Behind the new `embassy-tls` feature; `MqttConnectorBuilder::new` picks the transport from the URL scheme (`mqtt://` plain, `mqtts://` TLS) and gains `with_tls(TlsOptions)` (entropy, record buffers, SNTP server) and `with_credentials(username, password)` (MQTT CONNECT auth, both transports). The plain `mqtt://` path is untouched. The `embassy-mqtt-connector-demo` example gains a `tls` feature demonstrating the full flow against the repo's `dev/mosquitto` bench broker. ([aimdb-mqtt-connector](aimdb-mqtt-connector/CHANGELOG.md))
110115
- **Design 034 Phase 3 — sans-io KNX/IP tunneling engine shared by both transports (Issue #135, [review doc §3.7](docs/design/034-technical-debt-review.md)).** The entire tunneling lifecycle — CONNECT_REQUEST/RESPONSE handshake, TUNNELING_REQUEST/ACK sequence + pending-ACK bookkeeping, keepalive (CONNECTIONSTATE_REQUEST) scheduling, ACK-timeout sweeps, and reconnect-with-backoff — now lives **once**, in the new runtime-neutral `aimdb_knx_connector::tunnel` module (`no_std + alloc`, no tokio/embassy imports), driven as a poll-based state machine (events in, `Action`s out, `next_deadline()` for timer arming). `tokio_client.rs` (988 → ~530 lines incl. a new fake-gateway integration test) and `embassy_client.rs` (1,055 → ~450 lines) are reduced to socket shims; the previously untestable handshake/ACK/keepalive/reconnect paths now have 15 host-run unit tests plus a scripted localhost-UDP roundtrip test. Behavioral unifications: Embassy gains the 5 s CONNECT_RESPONSE timeout (previously waited forever), both shims reconnect on fatal socket errors, and the dead tokio-only per-publish ACK oneshot is dropped (it was always `None`); the CONNECT_REQUEST HPAI stays per-transport (`LocalEndpoint`: tokio = real bound address, Embassy = NAT mode). ([aimdb-knx-connector](aimdb-knx-connector/CHANGELOG.md))
111116

@@ -419,6 +424,7 @@ builder.configure::<Temperature>("sensor.temperature", |reg| {
419424
```
420425

421426
**Key naming conventions:**
427+
422428
- Use dot-separated hierarchical names: `"sensors.indoor"`, `"config.app"`
423429
- Keys must be unique across all records (duplicate keys panic at registration)
424430
- For single-instance records, any descriptive string works (e.g., `"sensor.temperature"`, `"app.config"`)
@@ -514,6 +520,7 @@ This release introduces **bidirectional connector support**, enabling true two-w
514520
### Modified Crates
515521

516522
See individual changelogs for detailed changes:
523+
517524
- **[aimdb-core](aimdb-core/CHANGELOG.md)**: Core connector architecture, router system, bidirectional APIs
518525
- **[aimdb-tokio-adapter](aimdb-tokio-adapter/CHANGELOG.md)**: Connector builder integration
519526
- **[aimdb-embassy-adapter](aimdb-embassy-adapter/CHANGELOG.md)**: Network stack access, connector support
@@ -523,6 +530,7 @@ See individual changelogs for detailed changes:
523530
### Migration Guide
524531

525532
**1. Update connector registration:**
533+
526534
```rust
527535
// Old (v0.1.0)
528536
let mqtt = MqttConnector::new("mqtt://broker:1883").await?;
@@ -533,6 +541,7 @@ builder.with_connector(MqttConnectorBuilder::new("mqtt://broker:1883"))
533541
```
534542

535543
**2. Make build() async:**
544+
536545
```rust
537546
// Old (v0.1.0)
538547
let db = builder.build()?;
@@ -542,6 +551,7 @@ let db = builder.build().await?;
542551
```
543552

544553
**3. Use directional link methods:**
554+
545555
```rust
546556
// Old (v0.1.0)
547557
.link("mqtt://sensors/temp")
@@ -552,6 +562,7 @@ let db = builder.build().await?;
552562
```
553563

554564
**4. Remove manual MQTT task spawning (Embassy):**
565+
555566
```rust
556567
// Old (v0.1.0) - Manual task spawning required
557568
let mqtt_result = MqttConnector::create(...).await?;
@@ -628,12 +639,14 @@ impl MyConnector {
628639
```
629640

630641
**Why this change?** The new trait-based architecture provides:
642+
631643
- ✅ Symmetry with inbound routing (`ProducerTrait``ConsumerTrait`)
632644
- ✅ Testability (can mock `ConsumerTrait` without real records)
633645
- ✅ Type safety via factory pattern (type capture at configuration time)
634646
- ✅ Maintainability (connector logic stays in connector crate)
635647

636648
**Migration checklist for custom connectors:**
649+
637650
- [ ] Add `spawn_outbound_publishers()` method to connector implementation
638651
- [ ] Call `db.collect_outbound_routes(protocol_name)` in `ConnectorBuilder::build()`
639652
- [ ] Call `connector.spawn_outbound_publishers(db, routes)?` before returning
@@ -654,6 +667,7 @@ See [Connector Development Guide](docs/design/012-M5-connector-development-guide
654667
### Added
655668

656669
#### Core Database (`aimdb-core`)
670+
657671
- Initial release of AimDB async in-memory database engine
658672
- Type-safe record system using `TypeId`-based routing
659673
- Three buffer types for different data flow patterns:
@@ -667,6 +681,7 @@ See [Connector Development Guide](docs/design/012-M5-connector-development-guide
667681
- Remote access protocol (AimX v1) for cross-process introspection
668682

669683
#### Runtime Adapters
684+
670685
- **Tokio Adapter** (`aimdb-tokio-adapter`): Full-featured std runtime support
671686
- Lock-free buffer implementations
672687
- Configurable buffer capacities
@@ -677,6 +692,7 @@ See [Connector Development Guide](docs/design/012-M5-connector-development-guide
677692
- Compatible with ARM Cortex-M targets
678693

679694
#### MQTT Connector (`aimdb-mqtt-connector`)
695+
680696
- Dual runtime support for both Tokio and Embassy
681697
- Automatic consumer registration via builder pattern
682698
- Topic mapping with QoS and retain configuration
@@ -686,6 +702,7 @@ See [Connector Development Guide](docs/design/012-M5-connector-development-guide
686702
- Uses `mountain-mqtt` for embedded environments
687703

688704
#### Developer Tools
705+
689706
- **MCP Server** (`aimdb-mcp`): LLM-powered introspection and debugging
690707
- Discover running AimDB instances
691708
- Query record values and schemas
@@ -703,12 +720,14 @@ See [Connector Development Guide](docs/design/012-M5-connector-development-guide
703720
- Clean error handling
704721

705722
#### Synchronous API (`aimdb-sync`)
723+
706724
- Blocking wrapper around async AimDB core
707725
- Thread-safe synchronous record access
708726
- Automatic Tokio runtime management
709727
- Ideal for gradual migration from sync to async
710728

711729
#### Documentation & Examples
730+
712731
- Comprehensive README with architecture overview
713732
- Individual crate documentation with examples
714733
- 12 detailed design documents in `/docs/design`
@@ -719,6 +738,7 @@ See [Connector Development Guide](docs/design/012-M5-connector-development-guide
719738
- `remote-access-demo`: Cross-process introspection server
720739

721740
#### Build & CI Infrastructure
741+
722742
- Comprehensive Makefile with color-coded output
723743
- GitHub Actions workflows:
724744
- Continuous integration (format, lint, test)
@@ -730,28 +750,33 @@ See [Connector Development Guide](docs/design/012-M5-connector-development-guide
730750
- Dev container setup for consistent development environment
731751

732752
### Design Goals Achieved
753+
733754
- ✅ Sub-50ms latency for data synchronization
734755
- ✅ Lock-free buffer operations
735756
- ✅ Cross-platform support (MCU → edge → cloud)
736757
- ✅ Type safety with zero-cost abstractions
737758
- ✅ Protocol-agnostic connector architecture
738759

739760
### Known Limitations
761+
740762
- Kafka and DDS connectors planned for future releases
741763
- CLI tool is currently skeleton implementation
742764
- Performance benchmarks not yet included
743765
- Limited to Unix domain sockets for remote access (no TCP yet)
744766

745767
### Dependencies
768+
746769
- Rust 1.75+ required
747770
- Tokio 1.47+ for std environments
748771
- Embassy 0.9+ for embedded environments
749772
- See `deny.toml` for approved dependency licenses
750773

751774
### Breaking Changes
775+
752776
None (initial release)
753777

754778
### Migration Guide
779+
755780
Not applicable (initial release)
756781

757782
---
@@ -763,6 +788,7 @@ Not applicable (initial release)
763788
AimDB v0.1.0 establishes the foundational architecture for async, in-memory data synchronization across MCU → edge → cloud environments. This release focuses on core functionality, dual runtime support, and developer tooling.
764789

765790
**Highlights:**
791+
766792
- 🚀 Dual runtime support: Works on both standard library (Tokio) and embedded (Embassy)
767793
- 🔒 Type-safe record system eliminates runtime string lookups
768794
- 📦 Three buffer types cover most data patterns
@@ -771,6 +797,7 @@ AimDB v0.1.0 establishes the foundational architecture for async, in-memory data
771797
- ✅ 27+ core tests, comprehensive CI/CD, security auditing
772798

773799
**Get Started:**
800+
774801
```bash
775802
cargo add aimdb-core aimdb-tokio-adapter
776803
```
@@ -779,7 +806,7 @@ See [README.md](README.md) for quickstart guide and examples.
779806

780807
**Feedback Welcome:**
781808
This is an early release. Please report issues, suggest features, or contribute at:
782-
https://github.com/aimdb-dev/aimdb
809+
<https://github.com/aimdb-dev/aimdb>
783810

784811
---
785812

Cargo.lock

Lines changed: 29 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

aimdb-bench/Cargo.toml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,10 @@ harness = false
4040
name = "b0_alloc_migration"
4141
harness = false
4242

43+
[[bench]]
44+
name = "b0_alloc_linkable"
45+
harness = false
46+
4347
[features]
4448
default = ["std"]
4549
# Gates `profiles`/`reports`/`harness` and their criterion/serde_json/
@@ -89,6 +93,7 @@ serde_json = { workspace = true, optional = true }
8993
# criterion/serde_json above, since that's the only bench that needs it.
9094
aimdb-data-contracts = { path = "../aimdb-data-contracts", default-features = false, features = [
9195
"alloc",
96+
"linkable",
9297
"migratable",
9398
], optional = true }
9499

@@ -124,3 +129,4 @@ critical-section = { version = "1.1", features = ["std"] }
124129
# defmt logger / panic-handler + embassy-time driver stubs for the host bench
125130
defmt = { workspace = true }
126131
embassy-time-driver = { path = "../_external/embassy/embassy-time-driver" }
132+
postcard = { version = "1.0", default-features = false, features = ["alloc"] }

aimdb-bench/README.md

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,8 @@ Plus two informational benches that exercise the full runner-driven pipeline.
2020
**Adapters covered:**
2121

2222
- **Tokio**`b0_alloc_tokio`, `b1_b2_tokio` (host).
23+
- **postcard `Linkable` codec**`b0_alloc_linkable` (host). This isolates the
24+
issue #177 `encode_into` seam; it is not a whole-connector allocation claim.
2325
- **Embassy**`b0_alloc_embassy`, `b1_b2_embassy`
2426
(host). These drive the real [`EmbassyBuffer`] backend via
2527
`futures::executor::block_on` over embassy-sync's poll methods — no
@@ -53,6 +55,9 @@ Always run from the workspace root (`/aimdb_ws/aimdb`).
5355
# B0 — allocation gate (buffer layer)
5456
cargo bench -p aimdb-bench --bench b0_alloc_tokio
5557

58+
# B0 — allocation gate (Postcard Linkable::encode_into codec seam)
59+
cargo bench -p aimdb-bench --bench b0_alloc_linkable
60+
5661
# B1 + B2 — latency (time/iter) and throughput (msgs/sec), one Criterion suite
5762
cargo bench -p aimdb-bench --bench b1_b2_tokio
5863

@@ -91,6 +96,7 @@ Criterion writes HTML reports to `target/criterion/`.
9196
`b0_alloc_tokio` does not use Criterion. It runs a fixed warmup + batch cycle and writes JSON results to `aimdb-bench/target/bench-results/b0_alloc_tokio.json` (the path is anchored to the crate dir, so it is the same regardless of the directory you run from).
9297

9398
**Measurement model:**
99+
94100
1. Create buffer + reader.
95101
2. Warmup ≥ 200 push → recv cycles (excluded from counters).
96102
3. Reset allocation counters.
@@ -103,6 +109,9 @@ The committed baseline lives in `data/baselines/b0_alloc_tokio.json`. When a cha
103109
104110
`b0_alloc_embassy` mirrors this against the Embassy buffer backend and writes `data/baselines/b0_alloc_embassy.json` — also **0 allocs/msg** across all three profiles, confirming the Embassy `poll_recv` path is allocation-free on the host. The on-target B3 bench (`examples/embassy-bench-stm32h5`) re-checks the same 0-alloc claim against the real embedded allocator.
105111

112+
`b0_alloc_linkable` warms up for 1,000 iterations, then measures 10,000 generated-shape postcard `Linkable::encode_into` calls into one stack buffer.
113+
The required result is **0 allocation calls and 0 allocated bytes**. It isolates the codec seam: `SerializedReader` still returns a boxed future, dynamic topics may allocate and connector implementations may copy payload ownership after the core pump lends them the scratch slice.
114+
106115
> **Embassy eager registration (design 039 F8/F9).** An Embassy `SpmcRing` reader registers its embassy `Subscriber` eagerly, at `subscribe()` time — matching Tokio's `broadcast` — so no separate priming step is needed before the first `push`.
107116
108117
**Noise reduction:** a `new_current_thread()` Tokio executor is used so there are no work-stealing threads and Tokio's scheduler does not allocate per-poll in the hot path.
@@ -127,5 +136,3 @@ Use them as a comparison point, not a regression gate. If they regress, `b0_allo
127136
- Criterion p99 can vary ±5–10% on noisy CI runners. Use p50 medians for trend comparisons.
128137
- Always specify `--release` or debug build consistently when comparing runs; optimizations differ by 5–50×.
129138
- `b_alloc_pipeline` uses a paced source: per-message pace tokens and notification channels. The coordination overhead is included in the measured window.
130-
131-

0 commit comments

Comments
 (0)