Dual-backend RNS architecture (Python + Swift) — atomic integration#82
Closed
torlando-tech wants to merge 120 commits into
Closed
Dual-backend RNS architecture (Python + Swift) — atomic integration#82torlando-tech wants to merge 120 commits into
torlando-tech wants to merge 120 commits into
Conversation
Remove reticulum-swift, LXMF-swift, and LXST-swift from Package.swift, and delete all voice/call code (CallManager, CallKitManager, AudioManager, CodecProfileInfo, the Views/Call directory, and the voice unit tests). Strip the voice integration points from ColumbaApp.swift, AppServices.swift, and MessagingView.swift. Remove voip from UIBackgroundModes. The app will not build on this commit — that is the intent for Phase 0. Phase 1 installs the BeeWare Python-Apple-support bridge and rewrites AppServices over it. See the migration plan at ~/.claude/plans/okay-please-explore-this-spicy-cloud.md and the PoC at ~/repos/columba-python-poc/. Voice returns in v2 once canonical Mark Qvist Python LXST is ported to iOS audio (AVAudioEngine replacement for PyAudio). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop in everything needed to embed canonical Mark Qvist Python RNS + LXMF on
iOS, sourced from the proven PoC at ~/repos/columba-python-poc/.
Frameworks/.gitkeep — Python.xcframework lands here
support/fetch-python.sh — pulls BeeWare 3.13-b13 (~110 MB)
support/fetch-wheels.sh — pulls iOS wheels for rns, lxmf,
cryptography 47.0.0, cffi 2.0.0,
pyserial into wheels-iphoneos/
and wheels-iphonesimulator/
support/README.md — pinned-version table, bump procedure
Sources/ColumbaApp/Python/PythonRuntime.swift — boots CPython, owns the GIL
(withGIL helper), releases the
GIL after init via PyEval_SaveThread
Sources/ColumbaApp/Python/PythonBridge.swift — serial-queue Swift<->Python boundary,
drainEvents() poll loop, identity_bytes
parameter for Keychain-fed identities
Sources/ColumbaApp/Python/ColumbaPython-Bridging-Header.h
— C shims around Py_None and the
PyConfig_SetString(&config, &config.home, ...)
exclusivity hazard
Sources/ColumbaApp/Python/Models/Py{Announce,Message,Conversation,LocalIdentity}.swift
— Swift value types the bridge emits;
Py-prefixed to avoid colliding with
deleted ReticulumSwift / LXMFSwift
types during the transition
app/rns_bridge.py — RNS.Reticulum + LXMF.LXMRouter
lifecycle, TCPClientInterface config,
lxmf.delivery announce handler,
opportunistic send/receive, status()
diagnostic, msgpack [name, stamp_cost]
app_data decoder, signal.signal
monkey-patch for off-main-thread RNS
init, singleton clear-down on stop()
This commit only lays down files; the Xcode pbxproj wiring (linking the
xcframework, declaring the bridging header, and adding the install_python
build phase from Frameworks/Python.xcframework/build/utils.sh) lands in
Phase 1b. App still doesn't build until that wiring is in place.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Edit Columba.xcodeproj via the xcodeproj Ruby gem (script committed for
reproducibility at support/configure-xcodeproj.rb). The script is idempotent —
re-running it is safe and only adds what's missing.
Changes to the ColumbaApp target:
- Drop 12 dead file refs for voice files deleted in Phase 0
- Drop SwiftPM remote refs + product deps for reticulum-swift, LXMF-swift,
LXST-swift
- Add Sources/ColumbaApp/Python/{PythonRuntime,PythonBridge}.swift and the
four Models/Py{Announce,Message,Conversation,LocalIdentity}.swift sources
- Add Sources/ColumbaApp/Python/ColumbaPython-Bridging-Header.h as a file ref
and set SWIFT_OBJC_BRIDGING_HEADER on Debug + Release configs
- Link + embed Frameworks/Python.xcframework with CodeSignOnCopy
- Add a Run Script build phase that sources install_python from the
xcframework's build/utils.sh — copies the platform-appropriate wheels
from wheels-iphoneos/ or wheels-iphonesimulator/ into <bundle>/app_packages/,
rsyncs the stdlib, and converts every Python .so extension into a
per-module .framework with codesign attached
- Add app/ as a folder-reference resource (rns_bridge.py + future Python
code lands at <bundle>/app/)
- EXCLUDED_ARCHS[sdk=iphonesimulator*]=x86_64 and ONLY_ACTIVE_ARCH=YES
(Debug) to dodge install_python's fat-simulator-build bug — captured in
the PoC writeup
- LD_RUNPATH_SEARCH_PATHS += @executable_path/Frameworks so dyld can find
the embedded Python framework at runtime
Verification: the Python bridge files (PythonRuntime.swift, PythonBridge.swift,
Models) now compile cleanly — `xcodebuild -target ColumbaApp build` only
reports `unable to resolve module 'LXMFSwift'` / `'ReticulumSwift'` from the
~56 untouched files. Phase 1c rewrites those over the new bridge.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adopts the module layout Columba Android is converging on for its
canonical-Python migration (feat/rns-dual-build branch, rns-api +
rns-backend-py + app + rns-host). iOS ships only the Python backend
(there's no analogue to reticulum-kt worth carrying), but the API split
and value-type discipline mirror the Android source.
Package.swift now declares four targets:
PythonBridge raw CPython embedding — runtime + GIL + bridging header
RNSAPI pure interface module: protocols + value-type models +
utility helpers. Mirrors rns-api/.
RNSBackendPy Python implementation of RNSAPI, wraps PythonBridge.
Mirrors rns-backend-py/.
ColumbaApp existing UI; will depend only on RNSAPI + RNSBackendPy
once Phase 3 migrates the call sites.
Moved PythonRuntime.swift / PythonBridge.swift / the bridging header out
of Sources/ColumbaApp/Python/ and into Sources/PythonBridge/.
Foundational types ported from rns-api this commit:
Sources/RNSAPI/Util/Aspects.swift ports Aspects.kt
Sources/RNSAPI/Util/HexExt.swift ports HexExt.kt (Data + String exts)
Sources/RNSAPI/Protocols/RNSError.swift ports RnsError.kt (no Parcelable —
iOS runs in-process)
Sources/RNSAPI/Protocols/BackendCapabilities.swift
ports BackendCapabilities.kt
(BackendID has only one case on iOS)
Sources/RNSAPI/Models/Identity.swift ports model/Identity.kt
Sources/RNSBackendPy/PythonRNSBackend.swift
skeleton; full impl in Phase 2
Next commits in dependency order:
Phase 1d — port the other 30 model types from rns-api/model
Phase 1e — port the 6 sub-protocols + RNSBackend umbrella
Phase 2 — port the 13 files of rns-backend-py to PythonBridge calls
Phase 3 — migrate the ~50 ColumbaApp files from
`import ReticulumSwift / LXMFSwift / LXSTSwift` to `import RNSAPI`
App still doesn't build until at least Phase 3 lands — current state is
the planned Phase 1b condition (Python embedded, UI files have dangling
imports for the removed Swift libs). build/ re-added to .gitignore.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ain)
Substantial progress toward making the existing Columba UI compile against
the new RNSAPI / RNSBackendPy / PythonBridge module split (mirrors Columba
Android's feat/rns-dual-build architecture). App still doesn't build — work
in progress, committing as a checkpoint.
Sources/RNSAPI/Compat.swift ~700 lines
Big compatibility-shim file holding stub types matching the AI-Swift API
shapes the existing UI references. Real bodies only where they don't need
the Python bridge (Keychain ops on Identity, hex helpers); everything else
is no-op or returns mock values. LXMFDatabase API mirrored exactly from
the deleted LXMF-swift surface (saveMessage(_:), getMessages(forConversation:),
etc.) so MessageRepository compiles without rewriting all its call sites.
Types stubbed/realized:
Identity (real: keys+hash+Keychain via CryptoKit+Security)
Destination, Link, Packet, Announce, AnnounceHandler
LXMessage (Data-backed content/title with String helpers)
LXMessageState, LXDeliveryMethod
LXMRouter (sendHook injection point for RNSBackendPy)
LXMRouterDelegate, LXMFDatabase, ConversationRecord, MessageRecord
ReticulumTransport, PathTable, PathEntry, InterfaceSnapshot
NetworkInterface protocol + TCPInterface / AutoInterface /
BLEInterface / RNodeInterface / MPCInterface stubs
BLEConnectionInfo (mirrors AI-Swift's expanded surface)
InterfaceState, InterfaceKind, InterfaceConfig
IconAppearance (with both fgColor/bgColor and foregroundColor/backgroundColor)
PropagationNodeInfo, PropagationState
LocationSharingManager, TelemetryPacket, LocationTelemetry, RadioConfig
LXMFError, MessagePackValue (Hashable, indirect cases), RequestReceipt
Import migration: 37 files in Sources/ColumbaApp/ migrated from
`import ReticulumSwift / LXMFSwift / LXSTSwift` to no imports — types live
in the same Xcode module so no `import` directive is needed.
Compile-guarded sources that don't apply to v1's TCP+announces+opportunistic
slice (preserved unchanged inside `#if FEATURE_X_ENABLED ... #endif`):
#if COLUMBA_NOMADNET_ENABLED — 8 files (Services + ViewModels + Views/NomadNet/*)
#if COLUMBA_RNODE_ENABLED — 7 files (Views/Settings/RNodeWizard/*, RNodeConfigSheet)
#if COLUMBA_BLE_ENABLED — 1 file (BLEDevicePickerSheet)
#if COLUMBA_MIGRATION_ENABLED — 5 files (Migration*.swift + OnboardingRestoreSheet)
#if COLUMBA_LOCATION_ENABLED — 1 file (LocationSharingManager)
#if COLUMBA_ONBOARDING_ENABLED — 6 files (Views/Onboarding/*)
Total: 28 files guarded. Each can be re-enabled by adding the
matching macro to OTHER_SWIFT_FLAGS=-D<FLAG> as its types come back
online in the relevant phase.
Module structure (Sources/):
PythonBridge/ — relocated CPython runtime + GIL helper + bridging header
RNSAPI/ — Compat.swift + Identity.swift + Util/{Aspects,HexExt}.swift
+ Protocols/{RNSError,BackendCapabilities}.swift
RNSBackendPy/ — PythonRNSBackend.swift skeleton (full impls coming)
ColumbaApp/ — unchanged structure, imports migrated
Build state: ~30 unique compile errors remain, all in AppServices.swift's
deep RNS-specific call paths (interface state machine, ratchet management,
Destination init shapes, RNodeInterface.disconnect, PathTable.removeAll).
These are mechanical fixes — mirror the AI-Swift API exactly on each
stub call. Next session continues the iteration.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Compiles, launches on simulators, and an opportunistic LXMF message sent
from one sim is received in the other sim's existing Chats UI — all
through the embedded canonical Mark Qvist Python RNS + LXMF stack. No
more reticulum-swift / LXMFSwift / LXSTSwift in the pipeline.
Wiring (new):
Sources/ColumbaApp/Services/PythonConfigWriter.swift
Serializes [InterfaceEntity] → RNS ConfigObj-format config file.
Drives Python's RNS.Reticulum(config_dir) off the user's saved
interfaces from InterfaceRepository — no host/port hardcoded in
source. Empty interface list = config with [reticulum]/[logging]
only; app comes up offline until the user adds an interface via
Settings → Manage Interfaces.
Sources/RNSBackendPy/PythonRNSBackend.swift
Wraps PythonBridge. Exposes start(StartParams), sendOpportunistic,
statusSnapshot, and an AsyncStream<PythonBridge.Event> drained at
200ms cadence into Columba's event plumbing.
AppServices.startPythonBackend
Now reads InterfaceRepository.getEnabledInterfaces() (App Group
UserDefaults suite), writes the RNS config via PythonConfigWriter,
boots PythonRNSBackend, hooks LXMRouter.sendHook to route outbound
LXMF through Python, and starts an event-drain task.
AppServices.persistInboundFromPython
Translates Python's `inbound` event into a synthetic LXMessage,
ensureConversation + saveMessage via MessageRepository, and posts
IncomingMessageHandler.messageReceivedNotification so the existing
ChatsViewModel / MessagingViewModel refresh paths pick it up. This
is the seam that makes inbound messages appear in the chat list UI.
RNSAPI/Compat.swift LXMFDatabase
Stub no-op replaced with a real in-memory store (conversations +
messages + peerIcons, thread-safe via NSLock). ChatsView, contacts
list, and messaging thread render from these maps until the SQLite
backing lands in a later phase.
App boot:
ColumbaApp.init now boots PythonRuntime.shared.start() before any
SwiftUI work — PyGILState_Ensure deadlocks otherwise.
RootView._initializeServicesOnce auto-creates a Reticulum identity
when COLUMBA_ONBOARDING_ENABLED is off (the onboarding feature flag
is currently off; without auto-create the smoke-test flow hits
AppServicesError.identityNotInitialized on first launch).
ColumbaApp's lxma:// URL handler grew a test-send dispatch:
lxma://test-send?to=HEX&content=... → NotificationCenter →
AppServices listens and calls backend.sendOpportunistic. For
Maestro/CLI smoke tests; not exposed in the UI.
AppServices.handlePythonEvent's announce branch auto-sends a
"[smoke] hello from <prefix>" reply to each peer the first time
we see its announce, gated by smokeTestAutoSentTo. Closed-loop
proof that announce → recall(identity) → opportunistic LXMF send →
delivery callback all work without UI interaction.
App ↔ Python bridge:
rns_bridge.py.start(...) no longer takes tcp_host/tcp_port — Swift
writes <config_dir>/config from PythonConfigWriter before calling
in, and Python just reads it. Reannounce thread schedules
delivery_destination.announce() at +2/+5/+15/+30s so the announce
doesn't get lost if the TCP interface hasn't come online yet.
PythonBridge.swift: types/methods promoted to `public` so
RNSBackendPy can import (same module under Xcode, but PythonBridge
is its own SwiftPM target for Swift-build tooling). Tuple args
dropped from 6 to 4 slots after host/port removal.
Compat layer growth:
LXMessage.content/title are now Data (matching AI-Swift API). Inout
overload of LXMRouter.handleOutbound added for MessagingViewModel.
sourceHash made `var` so persistInboundFromPython can construct
inbound messages. ConversationRecord gained destinationHash /
lastMessageTimestamp / lastMessagePreview aliases used by the
Chats UI. PathEntry fields made non-optional (sane defaults).
MessageRecord.state/method are now String (raw values) matching
the call sites' switch patterns.
New stub types added to keep the existing UI compiling without
surgery: CoreBluetoothBLEDriver, PropagationTransferState,
PeerLocation, SharingDuration (extracted out of macOS-only
PlatformCompat #else into always-available code), plus extra
BLEInterface / MPCInterface / RNodeInterface init shapes,
LXMRouter.setRatchetManager / setPropagationStampCost /
outboundPropagationNode / propagationStampCost / syncState /
syncFromPropagationNode, PathTable.pathUpdates,
ReticulumTransport.requestPath(for:),
PythonConfigWriter test-send / smoke-test plumbing.
Config-driven smoke test:
Seed an InterfaceEntity into the App Group UserDefaults suite
(group.network.columba.Columba, key com.columba.interfaces) as a
JSON array of one tcpClient entry pointing at your rnsd hub.
Restart the app; Python connects, announces, and the round-trip
works without any UI tap. See PythonConfigWriter for the full
config-format mapping.
What's not wired yet (deferred to next iteration):
- Settings → Manage Interfaces UI doesn't restart Python when the
user edits an interface. Requires app restart for changes to apply.
- Announce events still surface only via NotificationCenter; the
Contacts tab's path-table view will populate once PathTable is
backed by Python's RNS.Transport.destination_table.
- LXMessage outbound from MessagingViewModel flows through
LXMRouter.sendHook → Python, but doesn't reflect the real LXMF
message hash back to the UI (uses optimistic UUID for now).
- BLE / RNode / Multipeer interfaces emit a placeholder TCP line
in the config; the Swift-owned drivers wake the radio
independently in a later phase.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
UI showed "disconnected" even when Python's TCP socket was alive and
moving traffic, because the Compat-layer TCPInterface stub kept its
state hard-coded at `.disconnected`. This wires Python's authoritative
`RNS.Transport.interfaces[*].online` flag back to the Swift stubs every
two seconds, so NetworkStatusView and InterfaceManagementScreen render
correct online/offline badges.
rns_bridge.status() — gained a `section_name` field on each interface
entry (the config-section name, stable across the bridge). Also
added `status_json()` which returns json.dumps(status()), so the
Swift bridge can decode the whole snapshot with JSONDecoder
instead of walking a Python dict via the C API.
PythonBridge.status() — now returns `StatusSnapshot?` (Decodable
struct with nested InterfaceStatus list) instead of `[String:String]`
with a raw repr blob. JSON round-trip via PyUnicode_AsUTF8.
PythonRNSBackend.statusSnapshot() — return type widened to match.
AppServices.startPythonBackend now also:
- Seeds a Compat TCPInterface stub in `tcpInterfaces[entity.id]`
for each enabled TCP InterfaceEntity, starting at `.connecting`.
- Spins a `pythonStatusPollTask` Task that polls every 2 seconds.
- Each poll, `applyPythonInterfaceStatus` matches Python's interface
`section_name` to the InterfaceEntity (by recomputing the section
name PythonConfigWriter wrote) and flips the matching stub's
`state` to `.connected` / `.disconnected`. Logs the transition
via DiagLog `[PY] iface <section> -> <state> (rx=N tx=M)`.
AppServices.shutdown — cancels `pythonStatusPollTask` alongside the
existing pythonEventTask cancel.
Verified on two iPhone 17 Pro simulators against a fresh
TCPServerInterface hub on :4243: both sims log
`[PY] iface Smoke_Test_Hub-smoke- -> connected (rx=438 tx=609)` once
the TCP handshake completes, and the smoke round-trip continues to
land messages in the Chats UI as before.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
RNS has no hot-reload — changing the [interfaces] section needs a full
Reticulum re-init. Mirrors Columba Android's python flavor, which
publishes the same constraint via BackendCapabilities.InterfaceCaps.
hotReloadInterfaces = false and surfaces this same affordance.
UX:
Manage Interfaces toolbar trailing now shows "Apply & Restart"
(accent-colored, semibold) when any change is pending. Tapping
swaps the label to "Restarting…" with an inline ProgressView until
the new Reticulum instance is up. Accessibility hint: "Restarts
Reticulum to apply pending interface changes."
Add / edit / delete / toggle no longer auto-apply. Each mutation
marks `hasPendingChanges = true` and shows a transient success
toast ("Smoke Test Hub enabled — tap Apply to restart") so the
user sees their change was registered and what to do next. Avoids
surprise restarts while the user is mid-edit.
Wiring:
AppServices.restartPythonBackend() — cancels event drain + status
poll tasks, calls pythonBackend.stop(), drops the Compat
TCPInterface stub map, re-reads InterfaceRepository's enabled
list, and starts a fresh Python instance with the same cached
identity + display name. Public so the InterfaceManagement
view model can invoke it; the lxma://test-restart URL handler
invokes it for headless smoke tests.
InterfaceManagementViewModel.applyChanges() — body collapsed from
160 lines of per-interface-type orchestration (TCP/Auto/BLE/RNode/
Multipeer add/start/stop) down to `await
appServices.restartPythonBackend()`. Python is the single source
of truth for interface lifecycle now; the Swift-owned interface
code paths are dead and would have rotted under the Compat
stubs anyway.
AppServices caches `pythonStartIdentity` + `pythonStartDisplayName`
in `startPythonBackend()` so the restart path can re-invoke
without making the caller pass them in again.
rns_bridge.stop() — beefed up. Singleton clear alone wasn't enough;
RNS.Transport's class-level lists (destinations, interfaces,
path_table, destination_table, announce_handlers, identities)
survive `reticulum.exit_handler()` and cause
`register_delivery_identity()` to raise "Attempt to register an
already registered destination." on the second start. Drain those
too before clearing the singleton flags.
Verified via lxma://test-restart on the iPhone 17 Pro sim: initial
connect at T=0, restart triggered at T+10s, fresh "[PY] started
identity=… destination=…" at T+10.01s with the same Keychain
identity, and "[PY] iface Smoke_Test_Hub-smoke- -> connected" again
at T+12s — full TCP teardown + reconnect + announce cycle in
~2 seconds.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PythonConfigWriter already serialized the .autoInterface(AutoInterfaceConfig)
case (type, group_id, discovery_scope, etc.); AutoInterfaceConfigSheet
already collects the fields. The missing pieces were:
- com.apple.developer.networking.multicast entitlement. iOS 14+
silently drops outbound multicast UDP without it (RNS.AutoInterface
relies on IPv6 link-local multicast for peer discovery). Apple grants
it per-app via the special form at
https://developer.apple.com/contact/request/networking-multicast —
until that's approved the entitlement is honored on Simulator only
and ignored on real devices. Comment in the entitlements file
documents the requirement and Apple URL.
- applyPythonInterfaceStatus didn't log non-TCP interfaces. Added a
consolidated `[PY] interfaces=Name1:1,Name2:0,...` line emitted on
every snapshot key change so AutoInterface / RNode / etc. status
is visible alongside the TCP iface state. lastInterfaceSnapshotKey
suppresses repeated identical-state logs.
Verified on iPhone 17 Pro sim with seeded
[TCPClient 10.0.0.x:4243, AutoInterface group=columba-smoke]:
[PY] wrote config (469 bytes, 2 interfaces)
[PY] interfaces=Smoke_Test_Hub-smoke-:1,LAN_Auto-smoke-:1
Both interfaces report online from Python's RNS.Transport.
Note for testing: simctl spawn defaults write only reaches the sim-wide
plist at /data/Library/Preferences/group.<id>.plist, but apps actually
read from /data/Containers/Data/Application/<APP-UUID>/Library/
Preferences/group.<id>.plist. To seed App Group UserDefaults for an
automated smoke test, write to the per-app container path directly
(via xcrun simctl get_app_container <udid> <bundle-id> data) and kill
cfprefsd to drop its in-memory cache. The lxma://test-* URL handlers
remain the easier path for runtime-triggered tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Re-enable COLUMBA_NOMADNET_ENABLED and route page fetches through the
embedded CPython rather than reimplementing RNS Link/RequestReceipt
state machines in Swift against the Compat stubs.
rns_bridge.fetch_nomadnet_page(dest_hash_hex, path, timeout,
form_fields=None) -> dict
Establishes an RNS.Link to the destination, sends a request for
`path` (with msgpack-packed form_fields if provided), waits for
the response callback, tears the link down, and returns
`{ok, status, data, content_type}`. Synchronous from the Swift
caller's perspective; `threading.Event` waits on RNS's async
callbacks. Status string is one of `ok | no-path | link-failed |
request-failed | timeout | bad-hash | not-started`. Identifies
on the link before the request so stateful node apps see who
we are.
PythonBridge.fetchNomadNetPage(destHashHex:path:timeout:formFields:)
-> NomadNetFetchResult
Marshals the Swift dict to a Python dict, calls the bridge,
parses the response bytes out with PyBytes_AsStringAndSize.
NomadNetFetchResult.Status enum maps the Python status strings to
Swift cases.
PythonRNSBackend.fetchNomadNetPage(...) — thin proxy.
NomadNetBrowserService rewritten. The previous implementation
managed Link cache + RequestReceipt status streams + MessagePack
response unwrap, all against the Compat shim (which has stubs for
Link.request, RequestReceipt.statusUpdates, etc.). New version
takes a `PythonRNSBackend` instead of (transport, pathTable),
delegates fetch/submit/download to the bridge, keeps the 12-hour
Micron page cache on the Swift side. fetchPage / submitForm /
downloadFile public signatures unchanged so NomadNetBrowserViewModel
+ MicronParser don't need to touch their code.
NomadNetBrowserViewModel.init / NomadNetBrowserView.init drop the
(transport, pathTable) params and take a PythonRNSBackend instead.
ContactsView's .browseSite case updated.
Columba.xcodeproj Debug config now defines COLUMBA_NOMADNET_ENABLED;
the 8 NomadNet-related files (Service, ViewModel, BrowserView,
MicronDocument, MicronParser, MicronDocumentView,
MicronRenderContainer, MonospaceLineView, ZoomableScrollView)
compile into the app.
lxma://test-nomad-fetch?to=HEX&path=/page/index.mu URL handler
triggers a backend.fetchNomadNetPage from outside the UI; logs the
status + first 120 bytes preview via DiagLog. For Maestro / CLI
smoke tests.
Verified against a local nomadnet daemon on the user's main rnsd:
[TEST-NOMAD] result ok=true status=ok bytes=2755 preview=`c
The 2755 bytes are the nomadnet default index.mu (ANSI-art "Welcome
To Test Server N0!" banner + link rows to /page/nomadnet.mu and
/page/meshchat.mu) — valid Micron markup ready for MicronParser.
Multipeer interface entity / RNode interface entity still emit the
TCPClientInterface placeholder in PythonConfigWriter — those need
their own Swift↔Python bridge work, not in scope for this commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Network tab's filter chips (peers / audio / sites / relays / all)
and the announce list itself were both empty because Python's announce
events were dead-ending at NotificationCenter — the Compat PathTable
was a no-op stub returning an empty AsyncStream, so ContactsViewModel's
pathUpdates subscription never received anything.
rns_bridge.py now registers four per-aspect announce handlers
(lxmf.delivery, lxmf.propagation, nomadnetwork.node, lxst.telephony)
via RNS.Transport.register_announce_handler — one each, scoped via
`aspect_filter` so an LXMF announce and a NomadNet announce from the
same identity both surface. The event payload gained `aspect` and
`public_keys` fields; stop() iterates the list to deregister all
four cleanly.
PythonBridge.Event.announce gained `aspect` and `publicKeysHex`
associated values, parseEventList pulls them from the Python dict.
AppServices.handlePythonEvent.announce branch now builds a real
PathEntry from each announce — aspect mapped to detectedAspect /
isLXMFPropagationNode / isLXSTTelephony / isKnownDestination — and
calls pathTable.insert(_:). Smoke-test auto-send gated to
lxmf.delivery only so we don't try to LXMF a nomadnet node.
RNSAPI/Compat.swift PathTable rewritten from no-op stub to a real
in-memory store. NSLock-guarded `[Data: PathEntry]` map; insert(_:)
updates the map and broadcasts to all live pathUpdates continuations;
pathUpdates yields a fresh AsyncStream per subscriber and replays
the current snapshot so late subscribers don't miss prior announces.
Cancellation tears down the continuation via onTermination.
Verified end-to-end: with the iPhone 17 Pro sim seeded to talk to the
user's main rnsd, the log shows:
[PY] announce dest=0d1ddfcc8a820d2fe2fe1d4c895e18fc
aspect=nomadnetwork.node name="Columba Test Node"
[PY] announce dest=6ed9f03900a71bc2e2b6ddb65ca537a1
aspect=lxmf.propagation name="False"
…and the Contacts > Network tab renders the entries (the
NomadNetBrowserView at the top of the screenshot was reached by
tapping one — the navigation only works if the announce was actually
in the list). Filter chips now appear because
viewModel.networkAnnounces.isEmpty is no longer true.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three bugs surfaced once the parser carried `Bxxx background colors forward across lines (faf17e4): 1. Centering broke against the document, not the screen. A wide row (e.g. fr33n0w/thechatroom's 550-char trailing-whitespace line) pushed the VStack out to ~4600pt; centered shorter rows landed at the middle of *that* width — way past the viewport. Fixed by capturing the actual screen viewport via GeometryReader in MonospaceScrollContainer (mirrors Android's `Modifier.widthIn(min = viewportLineWidth)` from NomadNetBrowserScreen.kt:474) and wrapping each scroll-mode row in `.frame(minWidth: viewportWidth, alignment: alignment.swiftUI)`. 2. Row-to-row column alignment drifted by half a cell because Core Text's `textAlignment = .center` strips trailing whitespace when computing the centered offset. Lines with a trailing space centered as if one cell narrower than lines without — visible as the letter "T" of "the chat room" wandering in the ASCII art. UILabel now always renders left-aligned (paragraphStyle and textAlignment) and visual centering is the SwiftUI .frame's job. 3. SF Mono renders Block-Elements (▗▄▖▝▀▘▙▟ etc.) at slightly different pixel widths than ASCII spaces, so 85-char rows of mixed content didn't end up the same width. Bundled JetBrains Mono (Apache 2.0/OFL, Regular + Bold, ~270KB each) for the monospace renderer — every glyph in the file has advance=600 confirmed via fontTools, matching what Android already uses (MicronComposables.kt's `JetBrainsMonoFamily`). Falls back to the system font if the bundled one fails to load. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three Settings cards that were silently no-op through the Compat
stubs now actually drive Python's embedded RNS + LXMF. Plus a comment
documenting why Background Transport stays compile-guarded.
rns_bridge.announce(display_name) — updates
delivery_destination.set_default_app_data with msgpack-packed
[name_bytes, 0] (matching LXMF.LXMRouter.get_announce_app_data),
then calls delivery_destination.announce(). Returns {ok, reason}.
rns_bridge.set_propagation_node(hex_hash, stamp_cost) —
router.set_outbound_propagation_node + .set_propagation_node_stamp_cost.
Empty hex_hash clears the selection.
rns_bridge.propagation_sync(timeout) — calls
router.request_messages_from_propagation_node and polls
propagation_transfer_state until terminal (PR_COMPLETE / PR_NO_PATH /
PR_TRANSFER_FAILED) or timeout. Returns {ok, state, received_messages,
reason} where state mirrors LXMRouter.PR_* as lowercase strings.
PythonBridge.swift exposes:
- announce(displayName:) -> Bool
- setPropagationNode(destHashHex:stampCost:) -> Bool
- propagationSync(timeout:) -> PropagationSyncResult { ok, state, ... }
The State enum maps Python's state strings to typed cases.
PythonRNSBackend.swift adds matching proxy methods.
AppServices.sendAnnounce now routes through pythonBackend.announce
instead of the empty-Packet → empty-send Compat stub chain.
Manual Announce button + AutoAnnounceManager timer + interface-
added auto-announce all share this path.
AppServices.startPythonBackend reads `transport_enabled` from the
App Group UserDefaults suite (same key the Transport Mode toggle
persists to) and passes it to PythonConfigWriter, so the RNS
config emits `enable_transport = yes` when the user enables the
Advanced toggle. SettingsView's Transport Mode toggle now calls
appServices.restartPythonBackend() to pick the new value up
(same Apply & Restart UX as Manage Interfaces).
PropagationNodeManager.selectNode / clearSelection / syncNow now
route through pythonBackend.setPropagationNode +
.propagationSync instead of the no-op Compat LXMRouter stubs.
Sync result maps to PropagationTransferState.State so the
existing Settings → Delivery & Retrieval UI renders correctly.
ColumbaApp.swift gains lxma://test-announce?name=... and
lxma://test-prop-sync?node=HEX URL handlers for smoke tests.
SettingsView.swift docstring on the (compile-guarded) Background
Transport card explains why it stays off: the old wiring drove
reticulum-swift's TCPInterface.beginTunnelMode, which the Compat
layer doesn't provide. Bringing it back needs either Python in
the Network Extension or a BGProcessingTask+silent-push wake
architecture — Phase 2/3 work.
Verified on iPhone 17 Pro sim:
[ANNOUNCE] sent via Python (name="ColumbaSim-via-Announce-button")
→ sim's destination appears in `rnpath -t` with fresh expiry
[PY] wrote config (298 bytes, 1 interfaces) → /tmp/config:
enable_transport = yes
(toggled `transport_enabled=true` in App Group plist → restart picks it up)
[TEST-PROP-SYNC] result ok=false state=request_sent received=0
(link to local lxmd established, request sent, sync timed out at
30s because no pending messages for this identity — the call chain
is verified end-to-end; state is the correct expected behavior).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Identity swap was implemented but never exercised after the Python RNS
migration. Added a `lxma://test-identity-switch` URL handler that
runs the full swap path (createIdentity + switchToIdentity + AppServices
.switchIdentity) and logs the destination hash before/after so we can
verify Python actually rebooted with the new keys.
Verified on iPhone 17 Pro sim:
[PY] started identity=bafed434… destination=8bd52848… (initial)
[TEST-IDSWITCH] before destination=8bd52848775e37f9ef81e15e50dd0a35
[PY] started identity=cb54d9a4… destination=b49d1416… (after swap)
[TEST-IDSWITCH] after destination=b49d1416e1613e34fbee367fd4c7ead5
(changed=true)
A follow-up announce surfaces b49d1416… in rnsd's path table on the
same TCP port the original identity used — confirming the OS-level
socket persisted while Python's Reticulum singleton was torn down
and re-initialized with the new identity bytes.
No production-code changes — switchIdentity already calls shutdown +
initialize, both of which already route through PythonRNSBackend
correctly. This commit just adds the verification harness.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The privacy toggle was wired into IncomingMessageHandler.router(_:
didReceiveMessage:), but the Python migration routes inbound LXMF
events around that handler entirely — Python's _delivery_callback
emits a `PY] inbound` event that goes straight to
AppServices.persistInboundFromPython → MessageRepository.saveMessage.
So under the new stack, messages from non-favorited senders were
saving regardless of the toggle.
persistInboundFromPython now checks
UserDefaults.standard.bool(forKey: "block_unknown_senders") on
entry. If true, looks up the source's ConversationRecord and
requires isFavorite != 0 to proceed — matches the existing
IncomingMessageHandler semantics ("known" = explicitly favorited,
not merely "previously messaged"). Logs the drop as
`[PY] persistInbound BLOCKED source=<8hex> (block_unknown_senders
enabled)`.
Fails open if the DB lookup itself throws — surfacing a real
message is preferable to silently dropping mail because of a
transient I/O error.
ColumbaApp.swift gains lxma://test-inbound?from=HEX&content=...
URL handler — synthesizes a Python-style inbound event so the
filter can be exercised without a working peer (sim2's actual
outbound is currently blocked by an unrelated iOS-Python
_multiprocessing issue in LXMF.LXStamper).
Verified on iPhone 17 Pro sim:
block_unknown_senders=true:
[TEST-INBOUND] from=eb7cc16b… content="should-be-blocked"
[PY] persistInbound BLOCKED source=eb7cc16b
(block_unknown_senders enabled)
block_unknown_senders=false (same source):
[TEST-INBOUND] from=eb7cc16b… content="should-now-persist"
[PY] persistInbound saved msg=e656cd4c
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…nts)
First commit of the LXST voice-call pipeline. Exposes RNS.Link
operations through the Python bridge so the Swift LXST state machine
(lxst-swift Telephone actor, landing in commit 2) can drive call
lifecycle and audio framing without touching Python's GIL for the hot
path. Mirrors how Columba Android's lxst-kt sits on top of native
Kotlin code with msgpack wire compat to upstream Python LXST —
audio + state machine stays in the native runtime, Python is only the
underlying Link pipe.
rns_bridge.open_link(dest_hash_hex, aspect="lxst.telephony") —
initiate outbound RNS.Link to a destination. Returns int link_id
immediately; the `link_state=established` event fires async when
the Link handshake completes. `no-path` if RNS hasn't recalled
the peer identity yet (caller should retry after the next
announce).
rns_bridge.link_send(link_id, data_hex) — wrap bytes in a single
RNS.Packet and ship over an active Link.
rns_bridge.link_identify(link_id) — reveal local identity to the
remote so the remote's `link_identified` event fires.
rns_bridge.link_teardown(link_id) — close a Link from our side
(closed_callback fires `link_state=closed` on both peers).
New event types on the drain queue:
- link_state(link_id, state, reason, inbound)
state ∈ {establishing, established, closed}; inbound=true when
the remote opened the Link to us via our telephony destination.
- link_packet(link_id, data_hex)
opaque payload bytes received. hex-encoded for transit since
the event-marshalling layer is string-based.
- link_identified(link_id, identity_hash)
peer revealed identity via link.identify().
rns_bridge.start() now also registers a single inbound
`lxst.telephony` destination on the local identity with a
link_established_callback that allocates a fresh link_id and
synthesizes a `link_state=established inbound=true` event so the
Swift side sees one canonical state transition either way.
rns_bridge.stop() tears down all live Links + clears the
telephony destination so RNS.Reticulum can re-init cleanly on
identity swap / Apply & Restart.
PythonBridge.swift exposes openLink / linkSend / linkIdentify /
linkTeardown and gains three new Event cases. parseEventList
decodes link_packet bytes via a small Data(hexEncoded:) extension.
PythonRNSBackend.swift adds matching proxy methods.
AppServices.handlePythonEvent forwards the three new Link events
to NotificationCenter (ColumbaPythonLinkState /
ColumbaPythonLinkPacket / ColumbaPythonLinkIdentified) so
lxst-swift can subscribe without coupling AppServices to it
directly. Logs `[PY] link <id> state=<…>` for diagnosis.
lxma://test-link-open?to=HEX&aspect=<aspect> URL handler — for
smoke-test verification without the lxst-swift state machine
being wired yet.
Verified against the running nomadnet test daemon
(0d1ddfcc8a820d2fe2fe1d4c895e18fc, aspect=nomadnetwork.node):
[TEST-LINK] open ok=true linkId=1 reason=ok
[PY] link 1 state=establishing inbound=false
[PY] link 1 state=established inbound=false
Link came up in ~40ms.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ridge
Vendors lxst-swift (3,735 lines, 19 files) under Sources/LXSTSwift/ plus
its C dependencies COpus (libopus 1.5.2) and CCodec2 (codec2 1.2.0). The
package consumes the new RNSAPI Compat layer in lieu of the deleted
reticulum-swift surface; its previously-Packet-building Telephone.sendData
collapses to `link.sendBytes(_:)` so audio frames cross the Swift/Python
boundary via PythonRNSBackend.linkSend.
Build wiring:
- Package.swift now exposes only RNSAPI + LXSTSwift + their C deps; the
PythonBridge / RNSBackendPy / ColumbaApp targets that need Xcode's
bridging header live in pbxproj only (adding them to SwiftPM caused
Xcode's local-package wiring to compile them without the header).
- configure-xcodeproj.rb adds an XCLocalSwiftPackageReference at "." so
Xcode builds LXSTSwift+COpus+CCodec2 as native SwiftPM targets,
sidestepping the per-file pbxproj surgery ~380 C files would require.
RNSAPI additions to support lxst-swift:
- MsgPack.swift: minimal wire-format encoder/decoder (nil/bool/int/uint/
bin/str/array/map). Drop-in replacement for the deleted reticulum-swift
MessagePack module; the prior `unpackMsgPack` stub returned a raw dict
and never decoded anything.
- Link gains linkId, sendBytesHook, closeCallback, identifyCallbacks,
packetCallback storage + setCloseCallback/setIdentifyCallbacks/
setPacketCallback bridges. AppServices will install the hooks in a
later commit when wrapping live Python link_state events; Telephone
uses the callbacks during call lifecycle.
- TeardownReason enum, IdentifyCallbacks protocol — surface lxst-swift's
state machine expects.
- encrypt(_:)/decrypt(_:) pass-throughs — Python RNS.Link transparently
encrypts on the Packet build; iOS-side bytes are plaintext to/from
the bridge.
Type consolidation:
- InterfaceMode, RNodeConfig, PeerLocation, SharingDuration, and
PropagationTransferState now live canonically in RNSAPI/Compat.swift;
the rich ColumbaApp duplicates were removed (they were the only
callers, and Xcode now sees RNSAPI from the ColumbaApp target so
duplicates triggered "ambiguous type lookup").
Verification:
- `swift build` clean (LXSTSwift compiles standalone, no Python.h needed)
- xcodebuild Debug-iphonesimulator clean
- Installed + launched on two booted sims (iPhone 17 Pro / 17 Pro Max);
no startup crashes, Python runtime boots, OpenSSL legacy-provider
warning is unchanged pre-existing noise.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 0 stripped CallManager / CallKitManager / AudioManager + the
SwiftUI call screens along with the deleted AI Swift libs. They come
back here, restored from `2e3d4d5^`, with imports flipped from
`ReticulumSwift / LXMFSwift` to `RNSAPI` and the few API drifts
papered over inline.
Restored files (untouched in shape, only imports + a couple of
non-optional-displayName guards rewritten):
Sources/ColumbaApp/Services/CallManager.swift (852 LOC)
Sources/ColumbaApp/Services/CallKitManager.swift (348 LOC)
Sources/ColumbaApp/Services/AudioManager.swift (943 LOC)
Sources/ColumbaApp/Models/CodecProfileInfo.swift
Sources/ColumbaApp/Views/Call/{CallControlButton, CodecSelectionSheet,
IncomingCallScreen, PttButton, VoiceCallScreen}.swift
Compat-layer additions to make the restored CallManager wire cleanly:
- Link.setLinkEstablishedCallback(_:) + establishedCallback storage —
fired by AppServices when Python emits link_state(state=established)
for the matching linkId. CallManager defers "send AVAILABLE" until
this fires so the LRRTT handshake doesn't get raced.
- Hashing helpers (destinationNameHash, truncatedHash, identityHash)
moved from internal to public — PropagationNodeManager + a few
other app sites compute LXMF delivery hashes from path-table public
keys, exactly mirroring the Python upstream's truncation rules.
Build-graph cleanup (configure-xcodeproj.rb):
- RNSAPI is now a SwiftPM product dependency of ColumbaApp instead
of an inlined source tree. Compiling its files twice (once via
SwiftPM for LXSTSwift, once inline into the app target) produced
two distinct modules and call sites failed with "cannot convert
'ColumbaApp.Identity' to 'RNSAPI.Identity'". The script now strips
leftover inline refs and adds the product dep.
- Voice-file paths come out of DELETED_PATHS and into NEW_SWIFT.
- New group-routing branches for Views/Call and Models so the Xcode
navigator tree stays sensible.
Mechanical pass over the rest of ColumbaApp: 106 files gained an
explicit `import RNSAPI`. They used to see RNSAPI types automatically
(same compile target); now they consume RNSAPI as an external module
so the import has to be declared.
Verification:
- xcodebuild Debug-iphonesimulator clean
- Installed + launched on iPhone 17 Pro / 17 Pro Max sims; no crashes,
Python runtime boots, no new error logs.
- Telephony wiring (Python link_state events → Compat callbacks +
inbound link routing → CallManager.handleIncomingLink) is the
subject of commit 4 — at the end of commit 3 the UI compiles and
the call screen can render, but no Python event yet drives a
Compat Link object.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… (commit 4 of 5)
CallManager comes back online with full Compat-Link plumbing. Inbound
lxst.telephony links arriving from Python now actually surface in the
Telephone state machine and trigger CallManager.handleIncomingLink;
outbound calls flow through PythonRNSBackend.openLink.
Compat ReticulumTransport (RNSAPI):
- Adds four AppServices-installable hooks:
registerDestinationHook, unregisterDestinationHook,
registerDestinationLinkCallbackHook, initiateLinkHook.
- registerDestination / registerDestinationLinkCallback / unregisterDestination /
initiateLink delegate to the hook when set, otherwise no-op as before.
Keeps the build green during partial wiring and lets the Compat
transport stay value-typed instead of subclassed.
AppServices:
- Re-introduces a `callManager: CallManager?` property + initializes it
in the three places Phase 0 stripped the init from (cold start, warm
start with prior identity, and the migration path). Step 7b mirrors
the pre-Phase-0 sequencing — CallManager comes up before interfaces
so autoAnnounce includes the LXST telephony aspect.
- Adds `activeLinksByLinkId` + `destinationLinkCallbacks` state +
`dispatchInboundLink / dispatchOutboundLinkEstablished /
dispatchLinkClosed / dispatchLinkPacket / dispatchLinkIdentified`.
AppServices is @mainactor so the lookups don't need a lock; the
transport hooks are @sendable closures that hop back to main
before touching state.
- handlePythonEvent's linkState / linkPacket / linkIdentified arms now
dispatch to the Compat Link object in addition to the existing
NotificationCenter post (kept for debug-panel subscribers).
- openOutboundLink wraps Python.openLink → Compat Link, wiring the
Link's sendBytesHook to the linkSend(linkId:) call so audio frames
flow plaintext→bridge→encrypted Packet on the Python side.
Verified on both booted sims (iPhone 17 Pro / 17 Pro Max):
[INIT2] Step 7b: creating CallManager
[CALL] Registering LXST link callback for dest d86b6081...
[TEL_BRIDGE] registered dest-link callback for d86b6081
[INIT2] Step 7b done, telephonyDest=...
No crashes; Python event loop continues; no regression in non-voice
flows. Commit 5 will end-to-end the inbound-call leg: place a call
from sim1 to sim2, watch dispatchInboundLink fire on sim2, confirm
the call screen renders and one-way audio crosses.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the loop on the lxst-wiring batch. With this commit a call placed
from sim1 to sim2 fully establishes through the canonical Mark Qvist
Python RNS+LXST embedded in BeeWare — the full Reticulum LXST signal
exchange (AVAILABLE → LINKIDENTIFY → RINGING → CONNECTING → ESTABLISHED
+ PREFERRED_PROFILE) crosses the bridge.
Bridge wiring completed in this commit:
- Compat Link: identifyHook (parallels sendBytesHook), wired by
AppServices to PythonRNSBackend.linkIdentify(linkId:) so the
caller's `link.identify(identity:)` call propagates through Python
and the remote sees the proper IDENTIFY signal.
- Compat Link.identifyCallbacks: was weakly held, became strong.
Telephone.handleIncomingLink constructs a
`TelephoneIdentifyHandler(telephone: self)` inline with no caller-
held reference; under weak storage it deallocated immediately and
the inbound call silently stalled at AVAILABLE, never receiving
the identify-callback invocation when Python's link_identified
event arrived. Strong storage fixes it.
- app/rns_bridge.py: announce_telephony() function so the LXST
telephony destination announces alongside the LXMF delivery one;
sendAllAnnounces() invokes both. Without this, sim2 couldn't see
sim1's voice destination in its path table.
- URL test triggers: lxma://test-call?to=HEX[&profile=...] places
an outbound call through CallManager.initiateCall; lxma://test-answer
auto-accepts the ringing call. Mirrors the existing lxma://test-* set.
- CallManager auto-answer escape hatch keyed off env var
COLUMBA_AUTO_ANSWER=1. Reason: simctl openurl only delivers URLs
to the frontmost simulator window; sim2 stays backgrounded during
the sim1 → sim2 test so we can't drive the answer via URL. The
env var lets the smoke test reliably push sim2 past RINGING.
Smoke test (iPhone 17 Pro / 17 Pro Max sims, fresh boots):
sim1 → sim2: [TEST-CALL] to=e6abaf41... profile=qualityMedium
sim1: [TEL_BRIDGE] opened outbound link 1 → e6abaf41
sim2: [PY] link 1 state=established inbound=true
sim2: [CALL] handleIncomingLink
sim2: [TEL] sendData #1: 4B (AVAILABLE)
sim1: [TEL] handlePacket first4=81009103 (AVAILABLE)
sim1: (link.identify → linkIdentifyHook → Python linkIdentify)
sim2: [PY] link 1 identified=cb54d9a4
sim2: [CALL] Ringing from: cb54d9a4...
sim2: [TEL] sendData #2: 4B (RINGING)
sim2: [TEL] PREFERRED_PROFILE: Medium Quality
sim1: [CALL] Ringing from: 4e4f5bcb...
sim2: [CALL] COLUMBA_AUTO_ANSWER=1 — auto-answering
sim2: [TEL] answer(): sending CONNECTING, profile=Medium Quality
sim2: [TEL] sendData #3: 4B (CONNECTING)
sim2: [TEL] answer(): pipeline started, codec=opus
sim2: [TEL] answer(): sent ESTABLISHED
sim2: [CALL] establishedCallback fired
sim2: [AUDIO] startAudio() creating AudioManager
sim1: [TEL] handlePacket first4=81009105 (CONNECTING)
sim1: [TEL] handlePacket first4=81009106 (ESTABLISHED)
sim1: [AUDIO] startAudio() creating AudioManager
Audio frame round-trip (what we'd ideally see next, e.g. opus
"frame #1: hdr=0x01 audioBytes=N") doesn't fire on the simulator:
both sides reach [AUDIO] startAudio() but then log "deferring engine
start until didActivateAudioSession" because CallKit on the simulator
doesn't drive its audio-session-activation delegate the way a physical
device does (the sim has no real audio hardware to bind to). The
signaling path is end-to-end across the Python bridge; verifying
actual audio bytes traversing requires installing on Torlando's USB-
attached iPhone, which is a separate workstream from this 5-commit
batch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two env-var-gated hooks that let the sim↔iPhone test drive audio frames
across the bridge without requiring the user to navigate the iOS app
manually or to wait on CallKit's CXAnswerCallAction flow:
COLUMBA_TCP_HUB=<host>:<port>
Read in App/ColumbaApp.swift before AppServices.initialize. When set
AND no interfaces are configured yet, seeds a TCPClientInterface
record so a fresh-install build joins the same Reticulum hub as the
sim without manual onboarding. Address is environment-supplied at
launch (never committed in source — honors the LAN-IP-PII rule).
Existing COLUMBA_AUTO_ANSWER=1 path:
Already auto-answers incoming calls; this commit adds a second
effect in CallManager.startAudio() — when set, start the
AVAudioEngine immediately instead of deferring on
CallKit.didActivateAudioSession. The auto-answer path bypasses
CallKit's CXAnswerCallAction flow entirely, so CallKit never
activates its audio session and the engine would otherwise wait
forever. AudioManager.start() handles AVAudioSession activation
itself, so this is safe outside CallKit's orchestration.
Verification — sim → iPhone call (qualityMedium / opus) with both
hatches enabled on the iPhone:
iPhone (callee): [AUDIO] startAudio() engine started immediately
(session active=false bypass=true)
[TEL] TX frame #1: 1440 samples
Sim (caller): [TEL] handlePacket first: 20 bytes, first4=8101c410
[TEL] audioFrame #1: 20B, linkSource=true
[CALL] playReceivedAudio #1: 1440 samples
[CALL] playReceivedAudio #50: 1440 samples (3s later)
~16 fps matches the qualityMedium 60ms frame interval. Frames are
opus-encoded on the iPhone's AVAudioEngine input tap, crossed the
Python RNS.Link via PythonRNSBackend.linkSend (encrypted Packet over
TCP to the hub at the user-supplied address), arrived on the sim as
link_packet events, dispatched to lxst-swift LinkSource, decoded by
OpusCodec back to PCM samples ready for AudioManager playback.
The full LXST↔Python audio pipeline is verified end-to-end on a real
device. Codec2 negotiation was separately verified on sims using
`profile=bandwidthLow` (codec=codec2 selected on both sides).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the bidirectional audio-frame loop. With this commit the iPhone
places an outgoing LXST call to the sim's lxst.telephony destination and
streams opus frames over the Python RNS link; the sim receives, decodes
to 1440-sample PCM at the qualityMedium 60ms cadence, sustained 750+
frames over ~50s. Mirrors the previously-verified sim→iPhone direction.
Three smoke-test fixes the reverse run surfaced:
* Simulator carve-out in CallManager.startAudio() when bypassing
CallKit. The iOS Simulator's AVAudioEngine input node reports
sample rate 0 because there's no real mic — installTap throws an
uncaught Obj-C exception in `IsFormatSampleRateAndChannelCountValid`
and the whole app crashes. The sim never needs to capture mic for
the smoke test (it's always the callee receiving frames), so we
keep the deferred-engine-start path on simulator. Real-device
bypass (where the iPhone has actual hardware) is unchanged.
* teardownReason mapping in AppServices. Python emits
RNS.Link.teardown_reason as a numeric string (0=TIMEOUT,
1=INITIATOR_CLOSED, 2=DESTINATION_CLOSED — see Reticulum/Link.py).
The earlier code only matched the English names so every normal
hangup got logged as `.networkFailure`. Mapping now handles both
forms.
* lxma://test-announce was wired to sendAnnounce (delivery only); the
reverse test needs the telephony announce to also fire so the
iPhone can find the sim's voice destination in its path table.
Switched the URL handler to sendAllAnnounces so both go out together.
Adds one new smoke-test escape hatch:
COLUMBA_AUTO_CALL_TO=<hex>
Read in App/ColumbaApp.swift after AppServices init. When set,
polls the path table for up to 60s waiting for the target
destination to show up, then places an outgoing call via
CallManager.initiateCall. Used in the sim↔iPhone reverse test —
devicectl has no `open url` subcommand so we can't push a
`lxma://test-call` URL to the device; this env-var route does
the same thing through a startup hook instead.
Reverse-direction verification (sim1 callee, iPhone caller, Medium
Quality / opus):
iPhone: [AUTO_CALL] target found after 6s — placing call
[TEL_BRIDGE] opened outbound link 1 → d86b6081
[AUDIO] startAudio() engine started immediately (bypass=true)
[TEL] TX frame #1: 1440 samples
sim1: [PY] link 1 state=established inbound=true
[CALL] handleIncomingLink ... COLUMBA_AUTO_ANSWER auto-answering
[TEL] answer(): pipeline started, codec=opus
[TEL] handlePacket first: 30 bytes, first4=8101c41a
[TEL] audioFrame #1: 30B, linkSource=true
[CALL] playReceivedAudio #1: 1440 samples
[CALL] playReceivedAudio #750: 1440 samples (50s later)
The Compat-Link bridge now passes audio frames in both directions
through the canonical Python RNS+LXST stack. Full duplex audio across
two real devices is the only remaining variant; sim↔iPhone always
loses the sim→iPhone leg structurally because the simulator has no
audio input hardware.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phases 1-8c of the BLE interface port — mirrors Columba Android's
release/v0.10.x setup with Swift+CoreBluetooth in place of Kotlin+Chaquopy.
Architecture (3 layers):
Python IOSBLEInterface / IOSBLEDriver (in app/ble/)
Bridge PythonBLECallbackBridge (Swift→Python)
+ @_cdecl C-ABI shims (Python→Swift via ctypes)
Native SwiftBLEBridge + CBCentralManager + CBPeripheralManager
Swift→Python callbacks: Python registers callables into a slot dict in
rns_bridge.py; Swift looks them up on demand and invokes under the GIL.
Python→Swift calls: SwiftBLEBridge exports @_cdecl C-ABI shims (start,
stop, scan, advertise, connect, send, ...) that the Python driver binds
via ctypes.CDLL(None).
GATT service is wire-compat with Android + Linux ble-reticulum:
- 37145b00-442d-4a94-917f-8f42c5da28e3 (service)
- 5-byte fragment header [Type:1][Sequence:2 BE][Total:2 BE]
- 16-byte identity handshake on first RX write
Includes companion fixes outside the BLE port itself:
- applyPythonInterfaceStatus now updates Auto + BLE singletons in
addition to TCPInterface dict (UI was stuck disconnected).
- restartPythonBackend tears down + respawns Swift-side interface
stubs (NetworkStatusView was empty after Apply & Restart since
transport had no registered interfaces).
- PythonConfigWriter emits `type = IOSBLEInterface` for .ble rows.
Verified on Torlando's iPhone: app boots, deploys Python files, RNS loads
custom IOSBLEInterface, all 3 interfaces (TCP + Auto + BLE) reach
"connected" state through the full sync chain.
Design doc + executable plan:
~/Documents/Obsidian/columba-vault/80 Assistant/Memory/Columba-iOS/ble_interface_port_plan.md
~/.claude/plans/look-at-obsidian-80-shimmying-starlight.md
Companion reticulum-swift fork branches (not pushed; per memory rule):
fix/ble-peripheral-state-restoration
fix/ble-zombie-detector-background-aware
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ReticulumTransport's add*/get* methods were no-op stubs that discarded
every interface. NetworkStatusView calls transport.getInterfaceSnapshots()
which returned [], so the page rendered "No interfaces active" even when
TCP / Auto / BLE were all happily routing traffic.
Wire up a real lock-protected dict:
- addInterface / addAutoInterface / addBLEInterface / addMPCInterface
store by id, with type tracked alongside.
- removeInterface drops both.
- getInterfaceSnapshots maps each interface to an InterfaceSnapshot.
State is derived from `online` (the NetworkInterface protocol only
exposes `online`; concrete classes' `state` field stays as the source
of truth, applyPythonInterfaceStatus keeps both in sync).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
AppServices was auto-sending "[smoke] hello from <prefix>" to every peer the first time it saw their lxmf.delivery announce, ungated. Spammed every fresh client connecting to a shared rnsd. Other smoke-test escape hatches (COLUMBA_AUTO_ANSWER, COLUMBA_AUTO_CALL_TO) gate on env vars — this one was missed and shouldn't live in main app code at all. Removed the block and the `smokeTestAutoSentTo` dedup set that backed it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two bugs blocking AutoInterfacePeer / BLEPeer from showing up in
Network Status:
1. **JSON decode silently broke whole snapshot.** RNS sets
`iface.name = None` on dynamically-spawned peer interfaces
(AutoInterfacePeer, BLEPeerInterface), which serializes to JSON null.
PythonBridge.StatusSnapshot.InterfaceStatus.sectionName was a
non-optional String, so the entire snapshot decode threw and
getInterfaceSnapshots stayed empty. Fixed both sides:
- rns_bridge.status() coerces `name=None` to "" before serializing.
- InterfaceStatus has a permissive custom decoder that falls back to
empty/zero defaults per-field instead of failing the whole row.
2. **Compat transport had no path for python-discovered peers.** The
transport only tracked interfaces Swift explicitly added (TCP / BLE /
Auto stubs). RNS-spawned children weren't visible to
NetworkStatusViewModel. Added pythonAuxiliarySnapshots: AppServices'
status poll matches unmatched python interfaces by name prefix
(AutoInterfacePeer*, BLEPeer*) and pushes them through
ReticulumTransport.setPythonAuxiliarySnapshots; getInterfaceSnapshots
merges them into the output with the right type label + peerAddress.
Verified on device: a BLE peer now appears as
`auxiliary interfaces (1): py-aux:BLEPeerInterface[]` and renders in
the Network Status list.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The .entitlements file existed and was referenced in the project tree but no `CODE_SIGN_ENTITLEMENTS` build setting pointed at it for the ColumbaApp Debug/Release configs, so codesign produced a binary with ZERO custom entitlements (no app-group, no network-extension, no multicast). Verified: `codesign -d --entitlements -` on the built .app showed only `application-identifier` + `team-identifier` + `get-task-allow` before this change. Added `CODE_SIGN_ENTITLEMENTS = Sources/ColumbaApp/Resources/ColumbaApp.entitlements` to both TDBG and TREL build configs. Multicast Networking capability commented out — Apple's automatic signing refuses to embed it until the team is granted the capability at https://developer.apple.com/contact/request/networking-multicast. Other entitlements (com.apple.security.application-groups, com.apple.developer.networking.networkextension) now apply to the signed binary. AutoInterface uses BSD sockets via Python so it should still discover LAN peers without the entitlement; flip the multicast key back on after Apple approval if foreground discovery proves unreliable. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Before the app-group entitlement was wired, UserDefaults(suiteName: appGroupIdentifier) returned nil and InterfaceRepository fell back to .standard. After wiring, the suite returns a valid but empty defaults pointer, so interfaces appear to vanish even though the saved data still lives in .standard. Added a one-time migration in init: if the app-group store has no interface data AND .standard has some, copy it over. No-op once migrated. Doesn't help if iOS reset the data container as a side effect of the entitlement-identity change (in which case .standard is also empty), but does protect users who upgrade cleanly. Also added `diagnose_auto_interface` Python helper + `lxma://test-auto-diagnose` URL handler that introspects the running AutoInterface to surface peer-count / multicast bind state — used to trace why discovery isn't firing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
BLEConnectionsView reads via AppServices.getBLEConnectionInfos →
bleInterface.getConnectionInfos(), which was a `[]` no-op stub on the
Compat layer. The actual peer state lives in SwiftBLEBridge.shared
(our process-wide CB singleton); Compat's BLEInterface is a UI-only
stub with no behavior. Route the read through the bridge instead.
- Skip entries with no identityHashHex (those are in-flight central
connections that haven't completed the 16-byte handshake — not
something to render in the UI).
- Dedup by identity when a peer connects via BOTH central and
peripheral roles (would otherwise collide on
`BLEConnectionInfo.id == identityHex` in ForEach). Prefer the
peripheral entry since that's typically the established path with
higher MTU.
- Map RSSI to a coarse 4-bucket signalQuality matching the existing
BLEDevicePickerSheet thresholds (60/75/90 dBm).
- disconnectBLEPeer routes through the bridge too — Compat's
disconnectPeer was a no-op so taps on the disconnect button used
to do nothing.
Verified on device via `lxma://test-ble-peer-list`: SwiftBLEBridge.shared
returns 5 details (4 central handshake-in-progress + 1 peripheral
fully-connected with identity aec848fc).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
BLE Connections rows were showing "Signal unknown / nil dBm / Connected
0s forever" because:
- rssi was stubbed nil for peripheral-side entries (correctly, since
CB doesn't expose central-side RSSI to a peripheral), and the dedup
picked the peripheral entry every time so the rssi never appeared.
- connectionDuration was hardcoded to 0.
Three changes:
1. Added `connectedAt: Date` to BleConnectionDetails + each per-peer
GATT state object (BleGattClient, BleGattServerPeer). Re-stamped to
the post-handshake moment so the "Connected Xs" counter starts from
when the link is actually usable.
2. Periodic RSSI polling for central-side peripherals. Every 3s the
bridge calls `peripheral.readRSSI()` on each established
CBPeripheral; the result lands in the new `BleGattClient.rssi` via
the `peripheral(_:didReadRSSI:error:)` delegate. Also kicks a
one-shot read the moment the handshake completes so the UI gets a
sample without waiting a poll cycle.
3. AppServices.getBLEConnectionInfos now merges across dedup'd entries:
when a peer is connected via both central and peripheral roles and
we pick the peripheral entry as the representative, borrow the RSSI
from the central entry (which CB DOES sample) and use the earliest
connectedAt across the paths.
Verified on device: peer `aec848fc` now reports rssi=-69 on its central
entry; the merged BLEConnectionInfo for that peer in the UI gets
rssi=-69 + signalQuality=fair (|69| < 75 dBm threshold).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Brings the 18 main-only commits the long-running dual-backend branch had diverged past — most visibly the TCP client community-server wizard (#64), plus dark-mode maps (#65), granular auto-announce triggers (#70), the TCP hot-swap fix (#61), MicronParser formatting fix (#63), and the LXMF-swift 0.4.0 / reticulum-swift 0.3.0 dependency bump (#73). Conflict resolutions: - Dependency versions: KEPT the dual-backend pins (LXMF-swift 0.3.4, reticulum-swift 0.2.3, LXST-swift feat/transport-agnostic). #73 was a pure version bump (no Swift call-site changes), so main's code compiles at the older pins too, and keeping them avoids risking the SwiftRNSBackend that's verified against them. Bumping is deferred to a separate, Swift-backend- verified step. (project.pbxproj + Package.swift taken from this branch.) - InterfaceManagementViewModel.applyChanges(): kept the dual-backend's backend-agnostic appServices.applyInterfaceChanges() path; dropped main's legacy per-TCP connectTCPInterface loop. Full multi-TCP-per-entity reconciliation with the backend abstraction is deferred to the landing. - TCP wizard imports: on this branch InterfaceMode / InterfaceEntity live in RNSAPI (they were app-module types on main), so TCPClientWizard[ViewModel] now import RNSAPI instead of ReticulumSwift (whose own InterfaceMode collided). Wired main's 4 new source files into the ColumbaApp target. NOTE: main's 7 new test files came across on disk but are NOT yet added to the ColumbaAppTests target (they reference APIs that diverged here) — test-target reconciliation deferred. App builds clean: Debug (sim), Debug-Swift, Release. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Network Interfaces row for an RNode interface showed "disconnected" even while the RNode was connected and carrying traffic (the backend snapshot reported it online, e.g. `RNode-D8B5E1:1`). The status poll reads appServices.rnodeInterface.state, a Swift RNodeInterface stub. The real RNode runs as the Python IOSRNodeInterface, so the stub never reaches .connected on its own — applyPythonInterfaceStatus is supposed to mirror the backend's reported online state onto it (as it does for Auto and BLE), but the switch had `default: break` for .rnode, so RNode was never mirrored and the stub stayed .disconnected → the UI badge stayed disconnected. Add the .rnode case (section name already matches: sectionName(for:) yields `RNode-<id6>`, exactly what the backend reports). Same bug class as the Swift-backend indicator fix (0304d77). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Python-RNS migration (Phase 0) stripped the voice-call button + CallKit overlay from the conversation UI and never re-added them after LXST voice was re-integrated, so Views/Call/* (VoiceCallScreen, IncomingCallScreen, CodecSelectionSheet) existed but were presented nowhere — voice calls had no UI entry point. The CallManager mechanism itself works (proven via the lxma://test-call / test-answer hooks). Restored: - MessagingView: a phone button in the conversation toolbar → CodecSelectionSheet → CallManager.initiateCall. The conversation's LXMF hash isn't callable directly (telephony is a separate destination), so resolve the peer's <identity>.lxst.telephony hash via AppServices.telephonyHash(forPeerLxmfHash:) — recall the identity from the path table and derive the destination the same way CallManager builds our own. - MainTabView: app-root fullScreenCovers driven off callManager.callState so a call's UI shows from any tab and survives navigation — IncomingCallScreen while an incoming call rings (pre-answer; CallKit still covers system-level), VoiceCallScreen for outgoing/answered/ended. callState is read in onChange `of:` so @observable re-evaluates the covers reactively. Builds clean: Debug (sim + device), Debug-Swift, Release. Voice is still device-unverified end-to-end — placing/receiving a real call is the next check. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Drop a hardcoded /Users/<home>/… absolute path (machine-specific PII) in favor of Path(__file__).parent, matching the rest of the interop suite. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- fetch-wheels.sh: require explicit BLE_RETICULUM_LOCAL env-var for a local ble-reticulum checkout instead of implicitly picking up ~/repos/ble-reticulum (build reproducibility — matches RETICULUM_LOCAL / LXMF_LOCAL). - PythonBridge.swift: replace force-unwraps on CPython allocation calls (PyTuple_New / PyUnicode_FromString / PyDict_New / PyLong) with guard-let throwing / return-false, matching start()'s safe pattern — NULL under iOS memory pressure no longer crashes the process. - rns_bridge.py: drop the dead `terminal` set in the propagation-sync poll loop. - fetch-wheels.sh: correct the stale cryptography `dev1` comment. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 7be8fb4)
…ter 2) reset_identity called exit_handler() but, unlike stop(), never cleared RNS.Reticulum._Reticulum__instance (+ the exit-handler flags). Its docstring requires the caller to start() again afterwards, but that follow-on start() hit __init__'s 'Attempt to reinitialise Reticulum' guard, leaving the app unable to restart without a process relaunch. Mirror stop()'s cleanup. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tive (greploop iter 3) - reset_identity now also calls router.exit_handler() and tears down open _links / clears _telephony_destination (mirrors stop()), not just the Reticulum singleton — otherwise a follow-on start() spawns a 2nd LXMRouter racing the same lxmf-storage SQLite. - open-link path: move link.identify() to AFTER link_ready.wait() — identify sends an encrypted proof that a PENDING link can't, so identifying first silently no-op'd and the remote never learned our identity (breaks nomadnet stateful pages). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 59ed162)
…ter 4) _on_closed pops _links[link_id]; if the link closed before the entry existed the pop missed and open_link then inserted a permanently-zombie entry. Add to _links first, then wire callbacks — matching the inbound (_on_inbound_link) path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 3814427)
…rop test SwiftRNSBackend.propagationSync hardcoded receivedMessages: 0, so the UI's "N new messages" was always 0 after a sync. The router's syncState already carries the count (receivedMessages is incremented per retrieved message), so read it after the await instead of hardcoding 0. Verified via the interop harness (rnsd + lxmd + headless Sideband + sim): - RED reproduced: on the unfixed build, propagationSync returns received=0 with the full peer→node→sim round-trip executing. - Fixed build reaches the node (result ok=true) and reads syncState. Adds Tests/interop/test_propagation.py (the regression gate) + fixes silent harness rot: conftest's identity/delivery/propagation-node regexes still matched the old `[PY]` diag prefix (renamed to `[RNS]` during the dual-backend work), so the whole suite couldn't read the sim's identity. The received-count test does not pass in THIS host env because the local lxmd's TCPServer interface is misconfigured (`reachable_on 10.0.2.2:4242`) so propagated uploads never land on it — the test + fix are correct; the node needs fixing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…d-trip
The propagationSync received-count test was reaching iOS's sync but never
actually landing a message on lxmd, so it couldn't distinguish the fixed
backend from the bug (both reported 0). Three orchestration fixes, matching
the canonical propagation flow (both ends hear lxmd's announce → set it as
their propagation node → upload PROPAGATED → sync down):
* _peer_reach_node(): block until the Sideband peer has actually heard
lxmd's announce (recall + path), so its PROPAGATED upload has a node.
* re-set the peer's outbound propagation node AFTER lxmd is reachable —
the fixture sets it in start() before the path exists and swallows the
failure, leaving outbound_propagation_node unset.
* _sync_until_reachable() primes the path and retries past transient
noPath while the just-installed sim waits to hear lxmd's announce.
Verified red→green against the live harness: fixed backend → received>=1
(PASS); original hardcoded receivedMessages:0 → received=0 (FAIL at the
assertion). The test now genuinely catches the bug.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PythonBridge routes every call through one serial DispatchQueue. propagationSync blocks up to its timeout (default 60s) polling the router, so on that shared queue it stalled every other bridge call — announce, send, setPropagationNode — for the full duration. Move propagationSync onto a dedicated blockingQueue. The underlying Python poll releases the GIL between iterations (time.sleep), so short calls on the main queue acquire the GIL and run within ~0.5s while a sync is in flight. Two queues into CPython is safe — withGIL (PyGILState_Ensure/Release) serializes all Python access regardless of thread, the same pattern the BLE callback path already uses. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…locking The old test fired two separate lxma:// URLs (sync, then announce) and timed the announce from the first dispatch. On the Python backend that measured Maestro's per-openLink JVM-startup latency, not bridge blocking — the announce wasn't even dispatched until the ~10s sync had nearly ended, so the test couldn't tell the fixed backend from the buggy one. Replace it with an in-process probe (lxma://test-concurrency-probe): the app sets an unreachable propagation node, launches propagationSync WITHOUT awaiting it, waits 1.5s so it's mid-poll, then times a concurrent announce — all inside one Task, logged as [TEST-CONCURRENCY] announce_ms=.. sync_ms=... No Maestro dispatch latency, so the builds separate cleanly. Verified red→green on the Python backend: * fix (propagationSync on a dedicated blockingQueue): announce_ms=0, PASS * bug (propagationSync on the shared serial queue): announce_ms=9095, FAIL (both with sync_ms≈10600 — the announce stalls for the sync's whole window under the bug, returns instantly with the fix). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* PythonBridge.fetchNomadNetPage ran on the main serial queue, so a 30s NomadNet fetch stalled drainEvents + every other bridge call — the same starvation propagationSync was just fixed for. Route it to blockingQueue too. * _alloc_link_id incremented a counter with a non-atomic read-modify-write while being called from both the bridge thread and RNS callback threads, so two links could get the same id and overwrite each other in _links. Guard it with a dedicated lock (not the shared _lock — some callers already hold that, which would deadlock). * SwiftRNSBackend.stop() called eventContinuation.finish(), permanently terminating the AsyncStream; a later start() on the same instance dropped every event. Keep the continuation open across stop/start, matching PythonRNSBackend. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…s in CI * SwiftRNSBackend.propagationSync ignored its `timeout` parameter — syncFromPropagationNode() has only per-request timeouts, no overall deadline, so a stalled link/response blocked the caller forever. Race the sync against the timeout in a task group; the loser is cancelled. * support/fetch-wheels.sh pulled ble-reticulum from a bare repo URL (HEAD), making builds non-reproducible. Pin to a commit (origin/main@07d9413; the local checkout is 49 commits past v0.2.2, so a tag would regress). * .github/workflows/tests.yml built the Python-backend scheme without ever fetching Python.xcframework + the wheels, so the copy build-phase had nothing to copy. Add a fetch-python.sh + fetch-wheels.sh step before the builds. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* PythonRNSBackend.start() relied on the one-shot lazy `events` initializer to spin up the drain loop, but stop() cancels eventDrainTask — so after any stop/start (identity switch, reconnect) the loop never restarted and no Python events reached the host. Call startDrainLoop() in start() (no-ops if already running). * SwiftRNSBackend's per-link stateUpdates drain Task was fire-and-forget and untracked, so stop() (which only cleared `links`) left them running, holding the kept-open eventContinuation and yielding stale linkState events into the next session. Track them in a [Int: Task] and cancel on linkTeardown + stop(). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
reset_identity cleared the Reticulum singleton + _links but left RNS.Transport's process-global class state (destinations, interfaces, path tables, announce handlers, identities) populated. Its docstring requires the caller to call start() next — but that start() would raise "Attempt to register an already registered destination" (Transport.register_destination scans the stale Transport.destinations list), recoverable only by a process restart. stop() already did this teardown inline; reset_identity diverged and omitted it. Extract the shared block into _clear_transport_class_state() and call it from both, so the two teardown paths can't drift again. Note: resetIdentity is currently dead code (no callers in the app — identity switch goes through switchIdentity), so this hardens a latent bug rather than a live crash; grounded by mirroring the proven stop() path. Verified: py_compile clean; behaviour-preserving refactor of the live stop() teardown. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both stop() and reset_identity() iterated _links and called link.teardown() while holding _lock. link.teardown() can fire the _on_closed callback synchronously on the calling thread (once exit_handler() has stopped the transport's dispatch threads), and _on_closed does `with _lock: _links.pop(...)`. threading.Lock is non-reentrant, so that re-acquire would deadlock the bridge queue and the user couldn't disconnect/restart without killing the process. Snapshot _links + clear the dict inside the lock, then tear the snapshot down outside it — the same in-lock-snapshot / out-of-lock-side-effect ordering used elsewhere. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…solves
CI pinned Xcode 16.x, under which the embedded-Python (Columba) scheme fails to
build: the Python.framework clang module map isn't resolved ("module 'Python'
… not defined in any loaded module map file; … clang importer creation
failed"). Devs build locally on Xcode 26.x, where it's fine. Move CI to the
macos-26 runner and select the newest Xcode present (version-sorted) so CI
matches the local toolchain instead of pinning a stale major.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
xcodebuild test for the Columba scheme intermittently failed with "unable to resolve module dependency: 'LXMFSwift'/'ReticulumSwift'" for ColumbaAppTests on the fresh runner, even though the app build steps resolved the same local packages fine. Add an explicit -resolvePackageDependencies pass up front so the package graph is resolved once and reused by every build/test step. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Xcode 26's Explicitly Built Modules cannot resolve a @testable import of an app target that pulls in Swift packages: the dependency scan fails with "unable to resolve module dependency: 'LXMFSwift'" in ColumbaAppTests. Disabling explicit modules globally re-breaks the Python Clang module map / bridging header, so scope SWIFT_ENABLE_EXPLICIT_MODULES = NO to the test target only (all 4 build configs). The app keeps explicit modules so the Python bridging stays intact; the test target uses implicit modules so the @testable import resolves. Verified locally: build-for-testing -> TEST BUILD SUCCEEDED. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@testable import ColumbaApp exposes ColumbaApp's internal interface, which references LXMFSwift / ReticulumSwift / etc. The test target declared none of the app's Swift package products, so the test compile could not resolve those transitive modules: "missing required module 'LXMFSwift'" (implicit modules) / "unable to resolve module dependency: 'LXMFSwift'" (explicit). Mirror ColumbaApp's 6 package products (MapLibre, LXSTSwift, RNSAPI, SwiftBLEBridge, ReticulumSwift, LXMFSwift) onto ColumbaAppTests so the @testable import can resolve the full module graph. Generated via the xcodeproj gem (clean structural edit). Keeps the per-test-target SWIFT_ENABLE_EXPLICIT_MODULES = NO from the prior commit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
torlando-tech
pushed a commit
that referenced
this pull request
May 31, 2026
…ter 9) PR3 deleted 446 lines of tests for still-existing code (AudioRingBuffer, AudioManager, CallManager CallKit state machine), leaving the voice-call paths unverified (greptile #81). Restore all three and re-add the test target's explicit-modules config that lives on the combined branch (PR #82): - AudioRingBufferTests / AudioManagerConfigChangeTests: restored as-is; their APIs (AudioRingBuffer init/write/read/count; AudioManager init defaults + sampleRate/channels/frameTimeMs/samplesPerFrame/isActive) match current code, and they don't start the engine (no installTap) so they're simulator-safe. - CallManagerCallKitTests: adapted to PR2's refactor — import RNSAPI (was ReticulumSwift) and handleCallerIdentified now takes the delivery-hash Data (was an Identity); MockCallKitReporter matches the current CallKitReporting protocol verbatim. The outgoing path's startRingback() no-ops in tests (ensureToneOutput guards on audioSessionActivatedByCallKit=false). - pbxproj: SWIFT_ENABLE_EXPLICIT_MODULES=NO on ColumbaAppTests (4 configs) + the app's 6 package products, mirroring the CI-green config on PR #82, so the test target resolves @testable import ColumbaApp + the transitive packages. Verified by analysis (APIs/protocol match, syntax clean, sim-safe); the on-sim build-for-testing run is exercised by the stack's Tests CI on merge. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
torlando-tech
pushed a commit
that referenced
this pull request
May 31, 2026
…ter 9) PR3 deleted 446 lines of tests for still-existing code (AudioRingBuffer, AudioManager, CallManager CallKit state machine), leaving the voice-call paths unverified (greptile #81). Restore all three and re-add the test target's explicit-modules config that lives on the combined branch (PR #82): - AudioRingBufferTests / AudioManagerConfigChangeTests: restored as-is; their APIs (AudioRingBuffer init/write/read/count; AudioManager init defaults + sampleRate/channels/frameTimeMs/samplesPerFrame/isActive) match current code, and they don't start the engine (no installTap) so they're simulator-safe. - CallManagerCallKitTests: adapted to PR2's refactor — import RNSAPI (was ReticulumSwift) and handleCallerIdentified now takes the delivery-hash Data (was an Identity); MockCallKitReporter matches the current CallKitReporting protocol verbatim. The outgoing path's startRingback() no-ops in tests (ensureToneOutput guards on audioSessionActivatedByCallKit=false). - pbxproj: SWIFT_ENABLE_EXPLICIT_MODULES=NO on ColumbaAppTests (4 configs) + the app's 6 package products, mirroring the CI-green config on PR #82, so the test target resolves @testable import ColumbaApp + the transitive packages. Verified by analysis (APIs/protocol match, syntax clean, sim-safe); the on-sim build-for-testing run is exercised by the stack's Tests CI on merge. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
torlando-tech
pushed a commit
that referenced
this pull request
May 31, 2026
…ter 9) PR3 deleted 446 lines of tests for still-existing code (AudioRingBuffer, AudioManager, CallManager CallKit state machine), leaving the voice-call paths unverified (greptile #81). Restore all three and re-add the test target's explicit-modules config that lives on the combined branch (PR #82): - AudioRingBufferTests / AudioManagerConfigChangeTests: restored as-is; their APIs (AudioRingBuffer init/write/read/count; AudioManager init defaults + sampleRate/channels/frameTimeMs/samplesPerFrame/isActive) match current code, and they don't start the engine (no installTap) so they're simulator-safe. - CallManagerCallKitTests: adapted to PR2's refactor — import RNSAPI (was ReticulumSwift) and handleCallerIdentified now takes the delivery-hash Data (was an Identity); MockCallKitReporter matches the current CallKitReporting protocol verbatim. The outgoing path's startRingback() no-ops in tests (ensureToneOutput guards on audioSessionActivatedByCallKit=false). - pbxproj: SWIFT_ENABLE_EXPLICIT_MODULES=NO on ColumbaAppTests (4 configs) + the app's 6 package products, mirroring the CI-green config on PR #82, so the test target resolves @testable import ColumbaApp + the transitive packages. Verified by analysis (APIs/protocol match, syntax clean, sim-safe); the on-sim build-for-testing run is exercised by the stack's Tests CI on merge. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
torlando-tech
pushed a commit
that referenced
this pull request
May 31, 2026
…ter 9) PR3 deleted 446 lines of tests for still-existing code (AudioRingBuffer, AudioManager, CallManager CallKit state machine), leaving the voice-call paths unverified (greptile #81). Restore all three and re-add the test target's explicit-modules config that lives on the combined branch (PR #82): - AudioRingBufferTests / AudioManagerConfigChangeTests: restored as-is; their APIs (AudioRingBuffer init/write/read/count; AudioManager init defaults + sampleRate/channels/frameTimeMs/samplesPerFrame/isActive) match current code, and they don't start the engine (no installTap) so they're simulator-safe. - CallManagerCallKitTests: adapted to PR2's refactor — import RNSAPI (was ReticulumSwift) and handleCallerIdentified now takes the delivery-hash Data (was an Identity); MockCallKitReporter matches the current CallKitReporting protocol verbatim. The outgoing path's startRingback() no-ops in tests (ensureToneOutput guards on audioSessionActivatedByCallKit=false). - pbxproj: SWIFT_ENABLE_EXPLICIT_MODULES=NO on ColumbaAppTests (4 configs) + the app's 6 package products, mirroring the CI-green config on PR #82, so the test target resolves @testable import ColumbaApp + the transitive packages. Verified by analysis (APIs/protocol match, syntax clean, sim-safe); the on-sim build-for-testing run is exercised by the stack's Tests CI on merge. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
torlando-tech
pushed a commit
that referenced
this pull request
May 31, 2026
…ter 9) PR3 deleted 446 lines of tests for still-existing code (AudioRingBuffer, AudioManager, CallManager CallKit state machine), leaving the voice-call paths unverified (greptile #81). Restore all three and re-add the test target's explicit-modules config that lives on the combined branch (PR #82): - AudioRingBufferTests / AudioManagerConfigChangeTests: restored as-is; their APIs (AudioRingBuffer init/write/read/count; AudioManager init defaults + sampleRate/channels/frameTimeMs/samplesPerFrame/isActive) match current code, and they don't start the engine (no installTap) so they're simulator-safe. - CallManagerCallKitTests: adapted to PR2's refactor — import RNSAPI (was ReticulumSwift) and handleCallerIdentified now takes the delivery-hash Data (was an Identity); MockCallKitReporter matches the current CallKitReporting protocol verbatim. The outgoing path's startRingback() no-ops in tests (ensureToneOutput guards on audioSessionActivatedByCallKit=false). - pbxproj: SWIFT_ENABLE_EXPLICIT_MODULES=NO on ColumbaAppTests (4 configs) + the app's 6 package products, mirroring the CI-green config on PR #82, so the test target resolves @testable import ColumbaApp + the transitive packages. Verified by analysis (APIs/protocol match, syntax clean, sim-safe); the on-sim build-for-testing run is exercised by the stack's Tests CI on merge. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
torlando-tech
pushed a commit
that referenced
this pull request
May 31, 2026
…ter 9) PR3 deleted 446 lines of tests for still-existing code (AudioRingBuffer, AudioManager, CallManager CallKit state machine), leaving the voice-call paths unverified (greptile #81). Restore all three and re-add the test target's explicit-modules config that lives on the combined branch (PR #82): - AudioRingBufferTests / AudioManagerConfigChangeTests: restored as-is; their APIs (AudioRingBuffer init/write/read/count; AudioManager init defaults + sampleRate/channels/frameTimeMs/samplesPerFrame/isActive) match current code, and they don't start the engine (no installTap) so they're simulator-safe. - CallManagerCallKitTests: adapted to PR2's refactor — import RNSAPI (was ReticulumSwift) and handleCallerIdentified now takes the delivery-hash Data (was an Identity); MockCallKitReporter matches the current CallKitReporting protocol verbatim. The outgoing path's startRingback() no-ops in tests (ensureToneOutput guards on audioSessionActivatedByCallKit=false). - pbxproj: SWIFT_ENABLE_EXPLICIT_MODULES=NO on ColumbaAppTests (4 configs) + the app's 6 package products, mirroring the CI-green config on PR #82, so the test target resolves @testable import ColumbaApp + the transitive packages. Verified by analysis (APIs/protocol match, syntax clean, sim-safe); the on-sim build-for-testing run is exercised by the stack's Tests CI on merge. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft — opened to run full CI on the complete tree and stage the atomic merge.
This is the complete dual-backend integration (the
RnsBackendprotocol seam + embedded-Python and native-Swift backends). It is the union of the three review PRs:pr1-rns-core— RNS core + protocol seam (greptile 5/5)pr2-app-services— app services + BLE bridge (greptile 4/5)pr3-ui— UI screens + tests (greptile 3/5)Those were split path-scoped to stay under greptile's 100-file review limit. They cannot be merged to
mainincrementally — 102/111 modified files import the new RNSAPI module, so the intermediate PRs don't build the full app standalone (e.g.Package.swiftreferencesSwiftBLEBridge/app sources that live in sibling PRs), and merging one alone would leavemainnon-building. A buildable ≤100-file re-split was investigated and proven impossible. So the stack is merged atomically here, against the complete tree, where the build is whole.Notable fixes folded in beyond the original split
Twelve correctness fixes from the greptile loop on #79, each build- or harness-verified:
propagationSyncran on the bridge's serial queue (starved every other call) and hardcodedreceivedMessages: 0— both fixed and proven red→green on the interop harness; same serial-queue fix applied tofetchNomadNetPage.propagationSyncignored itstimeout;_alloc_link_idhad a non-atomic counter race;SwiftRNSBackend.stop()permanently finished its event stream; the Python drain loop didn't restart after stop/start; per-link state tasks leaked paststop().reset_identityleftRNS.Transportclass state populated (collided on re-register) and held_lockacrosslink.teardown()(re-entrant-lock deadlock) — both fixed, mirroringstop().ble-reticulumto a commit; CI now fetchesPython.xcframework+ wheels before building.Deferred
CallManageralready registers the inbound handler for the telephony hash; confirming whether it's a real gap needs a device inbound voice call (not blind-fixed to avoid a conflicting registration).🤖 Generated with Claude Code