feat(zephyr): platform services contract, IRadio completion, and native Zephyr HAL (M2 T1-T3) - #41
Open
amirna2 wants to merge 14 commits into
Open
feat(zephyr): platform services contract, IRadio completion, and native Zephyr HAL (M2 T1-T3)#41amirna2 wants to merge 14 commits into
amirna2 wants to merge 14 commits into
Conversation
Replace the horizontal, subsystem-by-subsystem M2-M8 decomposition with vertical tracer-bullet slices: each slice ships a usable, end-to-end testable capability through the real DeviceBuilder/Device/sendData API, usable the moment it lands. Slice 1 pulls the leaky-IRadio fix (formerly M6) to the front. IRadio exposes no TX/RX today, so Device is welded to concrete LoraRadio; hoisting the send/receive contract onto IRadio is the single change that lets the same Device run on Zephyr, making the library usable at the first slice instead of the last. Later slices thicken it: routing/relay, crypto/MIC, storage, inclusion, then display and wifi/portal. Assisted-by: Claude Code (Opus 4.8)
The previous plan still sequenced capabilities (leaf first, relay and crypto later) - horizontal layering in disguise. A full read of the library compile-closure shows no shortcut exists anyway: Device unconditionally requires storage + InclusionController, sendData is gated on inclusion, relays must be included to pass MIC verification, and app payloads are always AES+CMAC once included. New plan: M2 is ONE tracer bullet - the entire library, thin, validated by a working mini cluster (hub + relay + leaf) doing full ECDH inclusion, encrypted+CMAC telemetry across a multi-hop path, dedup, and reboot persistence. Feasible in one milestone because the measured Arduino coupling surface is only 9 files plus two backends, and the used Crypto-lib modules (AES/CTR/Curve25519/SHA256) contain zero Arduino references - they co-compile on Zephyr byte-identical. Gate A: Arduino MiniHub + Zephyr relay + Zephyr leaf (live wire-compat). Gate B: role swap with a Zephyr hub. Thickening milestones follow the bullet, each verified on the running cluster. Assisted-by: Claude Code (Opus 4.8)
…s first, zero arch changes - Hardware is XIAO ESP32-S3 exclusively, both hats (drop Heltec; Arduino side uses the existing XIAO preset), so wire-compat is proven on identical hardware. - Verification ladder made explicit and strictly ordered: V0 = the FULL library compiles for Zephyr/XIAO; V1 = every applicable existing unit test suite runs ON the board (co-compiled Unity + same test sources, mirroring the existing test_seeed_xiao_esp32s3 practice; 10 of 13 suites apply, display/wifi suites are compiled out); V2 = the mini cluster gates. - Porting principle locked: close to zero architectural changes. The concrete HAL classes are the platform contract - Zephyr backend .cpp for the same LoraRadio and EEPROMStorage classes, guarded members only where a HAL header holds platform types. Drop the previously recommended IRadio hoist / PacketRouter injection; Device, PacketRouter and all interfaces stay unchanged. Assisted-by: Claude Code (Opus 4.8)
…2 cluster) Task-by-task plan against the tracer-bullet roadmap: Options.h shims, guarded couplings, EEPROM facade over NVS (EEPROMStorage.cpp unchanged), LoraRadio Zephyr backend (same class), pinned Crypto co-compilation, module glue + skeleton app (V0), on-target Unity runner for the 10 applicable suites (V1), hub/standard cluster examples and gates (V2). Assisted-by: Claude Code (Opus 4.8)
Defines exactly what the portable library consumes from the platform (kernel services, chip id, console, radio contract = concrete LoraRadio API, storage contract = Arduino EEPROM API, crypto, logging) and how each is provided per platform: structural separation, platform code in platform-owned files, no new ifdefs in logic. Includes the complete file-touch inventory with Arduino-diff status and the four decisions requested before any code changes. Assisted-by: Claude Code (Opus 4.8)
…docs Replace the Arduino-emulation approach (Options.h shims, EEPROM facade, guarded same-class backends) with completing RadioMesh's existing clean architecture, per review: - Core (src/common, src/core, src/framework) becomes 100% platform-free and compiles universally; kernel services go through a new narrow RadioMeshPlatform contract implemented per platform. - IRadio is completed with the TX/RX surface; Device and PacketRouter consume IRadio*/IByteStorage* via platform factories (PacketRouter gains setRadio, mirroring its existing setCrypto pattern). - Zephyr implements the contracts natively in ports/zephyr/hal (ZephyrPlatform, ZephyrLoraRadio, ZephyrNVSStorage over NVS - no EEPROM emulation); Arduino implementations move verbatim into src/hardware/src/platform/ArduinoPlatform.cpp. - Naming unified to the project convention: PascalCase C++ files and classes with inc/<domain>/ + src/<domain>/ layout, everywhere including ports/. Spec rewritten; implementation plan rev. 2 (task-per-seam, per-task approval gates); tracer-bullet roadmap principles/work items updated. V0/V1/V2 gates unchanged. Assisted-by: Claude Code (Opus 4.8)
…y core Add the platform services contract (common/inc/platform/RadioMeshPlatform.h: millis, delay, random, randomBytes, chipId, consoleInit, logWrite, and the loraRadio/byteStorage composition factories) and convert every core call site to it. The portable core (common/, core/, framework/) now contains no platform calls: the only platform include left is Options.h's Arduino branch, the sanctioned build switch. - ArduinoPlatform.cpp (hardware layer): Arduino implementations; the simpleRNG analog-entropy source, the Serial console bring-up, and the ESP32 efuse chip id move here verbatim from core files. RNG seeding is now once-per-boot from analog entropy instead of once per generated ID (same per-device uniqueness, fewer analog reads). - HostPlatform.h: inline host reference implementation, structurally selected by the contract header for builds that are neither Arduino nor Zephyr (keeps native tests linking; seed for a future ports/host). - Logger.h: output now sinks through RadioMeshPlatform::logWrite, removing the ARDUINO/__ZEPHYR__ branches from core. - Utils: getRandomBytesArray collapses to one portable path; simpleRNG leaves the public RadioMeshUtils namespace (no external callers). - KeyManager: seed derived from RadioMeshPlatform::chipId(); dead RNG.h include dropped. RoutingTable/InclusionController/DeviceBuilder/ EncryptionService/AsyncDevicePortal converted likewise. - Also resolves two pre-existing warnings surfaced by the zero-warnings gate: Device ctor init-list order (-Wreorder) and loadedState initialization (-Wmaybe-uninitialized). Gates: pio test -e native 4/4; seeed_xiao_esp32s3 and heltec_wifi_lora_32_V3 SUCCESS with zero warnings in RadioMesh code (remaining warnings are vendor: ESP32 core uart, U8g2); core purity grep clean. Assisted-by: Claude Code (Opus 4.8)
Assisted-by: Claude Code (Opus 4.8)
IRadio declared setup/telemetry only, so Device and PacketRouter reached past it to the LoraRadio singleton and the EEPROMStorage concrete class. Hoist the observed TX/RX call surface into IRadio (setParams, sendPacket, startReceive, readReceivedData, checkAndClearRx/TxFlag, getRadioStateError); LoraRadio now overrides them with no implementation changes, and its interface docs live on IRadio. - Device holds IRadio*/IByteStorage*, obtained via the RadioMeshPlatform::loraRadio()/byteStorage() factories, and hands the radio to the router via setRadio() once params are applied. - PacketRouter gains setRadio(IRadio*) (mirroring setEncryptionService) and a null guard in sendPacket; the LoraRadio::getInstance() call and the hardware include are gone from the core. - DeviceStorage.h includes IByteStorage.h instead of EEPROMStorage.h. - EEPROM sizing (ByteStorageParams(EEPROM_STORAGE_MAX_SIZE)) moves into the Arduino byteStorage() factory: sizing is port business, and the core no longer references EEPROM_STORAGE_MAX_SIZE. Core now includes no hardware/ headers beyond the RM_NO_*-guarded display/wifi ones in Device.h. Gates: native tests 4/4, seeed + heltec builds SUCCESS with zero RadioMesh warnings, purity grep clean. Assisted-by: Claude Code (Fable 5)
…pen items Assessment of drivers/lora/loramac-node/sx12xx_common.c: the Zephyr driver is interrupt-driven at every layer; blocking lora_send is a k_poll_signal wrapper over the same TxDone interrupt. To mirror Arduino's non-blocking startTransmit semantics, ZephyrLoraRadio uses lora_send_async and treats the k_poll_signal as the tx flag, with a 2x lora_airtime() deadline for the timeout path (the driver raises no signal on TX timeout) and a mandatory RX-cancel before send (driver-enforced modem exclusivity). PHY-CRC-corrupt frames are dropped in-driver; protocol CRC32 + MIC still validate end-to-end. Also resolved: NVS is direct-write with commit as documented no-op (writes are atomic and persistent; header zephyr/kvss/nvs.h), and storage_partition is verified present in the XIAO partition table (partitions_0x0_amp_4M.dtsi, partition@3b0000) - no overlay change. Assisted-by: Claude Code (Fable 5)
…ephyr/hal Native Zephyr implementations of the two porting contracts, mirroring the Arduino counterparts structurally (getInstance() singletons, PascalCase, inc/<domain> + src/<domain> layout): - ZephyrPlatform.cpp: RadioMeshPlatform over kernel services (k_uptime_get_32, k_msleep, sys_rand_get, hwinfo_get_device_id, printk log sink; console needs no init on Zephyr). Factories return the two implementations below. - ZephyrLoraRadio: IRadio over the Zephyr LoRa API (DT lora0; devicetree owns pins, pinConfig ignored). Preserves the Arduino non-blocking flag model per the spec's "Zephyr TX model": sendPacket cancels async RX (driver-enforced modem exclusivity) then lora_send_async with a k_poll_signal as the tx flag; checkAndClearTxFlag polls the signal with a 2x lora_airtime() deadline supplying RM_E_RADIO_TX_TIMEOUT (the driver raises no signal on TX timeout) and force-releases the modem. RX is lora_recv_async: the callback buffers packet + RSSI/SNR under a spinlock and the driver auto re-arms. CR 4/7 + preamble 8 pinned to the D5 wire contract (absorbed from lora_phy.h, which T7 retires). - ZephyrNVSStorage: IByteStorage over NVS (zephyr/kvss/nvs.h) on the DT storage_partition (present in the XIAO partition table). Reserved keys map to fixed ids, others FNV-1a hash into a dynamic range. NVS writes are atomic and persistent: commit() and defragment() are documented no-ops; clear() remounts as nvs_clear invalidates the mount. Not yet compiled by any target: the Zephyr build wires these in at T5 (V0); PIO gates unaffected (ports/ invisible) but verified green. Gates: native tests 4/4, seeed + heltec SUCCESS, zero RadioMesh warnings, purity grep clean, clang-format clean on all five files. Assisted-by: Claude Code (Fable 5)
The T3 assessment analyzed drivers/lora/loramac-node, but the build has used CONFIG_LORA_MODULE_BACKEND_NATIVE since M1 (loramac-node is deprecated). Re-verified against drivers/lora/native/sx126x/sx126x.c: - Confirmed unchanged: interrupt-driven end to end (DIO1 -> workqueue), modem exclusivity (-EBUSY; RX must be cancelled before TX), async RX auto re-arm, PHY-CRC-corrupt frames dropped in-driver. - Corrected: the native backend raises the k_poll_signal on BOTH TxDone (result 0) and chip-level TX timeout (result -ETIMEDOUT, 10 s), so checkAndClearTxFlag now maps -ETIMEDOUT to RM_E_RADIO_TX_TIMEOUT as the primary timeout path; the 2x lora_airtime() deadline is demoted to a backstop for the deprecated backend, which raises nothing on timeout. Spec and plan updated to describe both backends accurately. Assisted-by: Claude Code (Fable 5)
Drop loramac-node from the west manifest; the build has used the in-tree native SX126x driver since M1 and compiles no sources from it. The devicetree auto-enables the legacy CONFIG_LORA_SX126X symbol whose Kconfig select otherwise pins the module, so the example configs force it off. Verified: west update + full tx and rx builds SUCCESS with the module unregistered, zero warnings. Comments and governing docs no longer reference the removed backend. Assisted-by: Claude Code (Fable 5)
The Zephyr port design documents (platform interface design, milestone 0 decisions) are durable architecture records and move to docs/architecture/. Session working documents (plans, handoffs) are development-process artifacts with no value to library users: they leave the tree and docs/superpowers/ is now gitignored (kept locally for ongoing work). Assisted-by: Claude Code (Fable 5)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
First three tasks of the M2 tracer bullet (mini cluster on Zephyr/XIAO), completing the RadioMesh architecture so the core compiles universally and platform code lives only in porting layers.
d5a1ae2): newsrc/common/inc/platform/RadioMeshPlatform.hdeclaring the kernel services the core needs (millis, delay, random, randomBytes, chipId, consoleInit, logWrite) plus composition factories (loraRadio(),byteStorage()). Arduino implementations move verbatim tosrc/hardware/src/platform/ArduinoPlatform.cpp; an inline host implementation keeps native tests running. Every Arduino call site incommon/,core/,framework/is converted — the core no longer includesArduino.hanywhere.e5d79ad): the TX/RX surface that existed only on the concreteLoraRadio(setParams,sendPacket,startReceive,readReceivedData,checkAndClearRx/TxFlag,getRadioStateError) is hoisted ontoIRadio.Devicenow holdsIRadio*/IByteStorage*obtained via the platform factories, andPacketRoutergetssetRadio(IRadio*)injection — the last core-to-hardware call (LoraRadio::getInstance()inPacketRouter.cpp) is gone. Core includes nohardware/headers beyond theRM_NO_*-guarded display/wifi ones.a0edb98,91b74e4):ports/zephyr/hal/gainsZephyrPlatform.cpp(kernel services overk_uptime_get_32/sys_rand_get/hwinfo/printk),ZephyrLoraRadio(IRadio over the Zephyr LoRa API, preserving the Arduino non-blocking flag model:lora_send_asyncwith ak_poll_signalas the tx flag, spinlock-guarded RX handoff, D5 wire contract pinned), andZephyrNVSStorage(IByteStorage over NVS on the devicetreestorage_partition).a6c3de7): the unused legacy LoRa backend module is removed fromwest.yml(the build has used the in-tree native SX126x driver since M1); the example configs force off the devicetree-auto-enabled legacy driver symbol that otherwise pinned it.Related Issue(s)
N/A — milestone work tracked in
docs/superpowers/plans/2026-07-19-zephyr-port-m2-implementation.mdanddocs/superpowers/specs/2026-07-19-zephyr-platform-interface-design.md.Type of Change
Testing
pio test -e native: 4/4 passing (run per commit).pio run -e seeed_xiao_esp32s3and-e heltec_wifi_lora_32_V3: SUCCESS with zero RadioMesh warnings (run per commit).src/common,src/core,src/frameworkis clean; nohardware/incincludes remain in core outside theRM_NO_*-guarded ones.west updateplusmake build ROLE=txandROLE=rxSUCCESS after the manifest change (full compile, link, and ESP32-S3 image; zero warnings).Screenshots
N/A
Additional Notes
IRadiogained pure virtual methods — source-breaking only for externalIRadioimplementers (none known); the application-facing API (DeviceBuilder,IDevice, presets, callbacks) is unchanged, and wire format/crypto are untouched. Remaining M2 tasks: T4 crypto sources into the Zephyr build, T5 full-library Zephyr build (V0), T6 on-target unit suites (V1), T7 mini-cluster gates (V2).Assisted-by: Claude Code (Fable 5)