Skip to content

alt_core_api: Introduce a IDL/code-generation for compatible API headers#181

Open
soburi wants to merge 2 commits intozephyrproject-rtos:nextfrom
soburi:alt-api
Open

alt_core_api: Introduce a IDL/code-generation for compatible API headers#181
soburi wants to merge 2 commits intozephyrproject-rtos:nextfrom
soburi:alt-api

Conversation

@soburi
Copy link
Copy Markdown
Member

@soburi soburi commented Apr 29, 2026

Summary

This adds a protoc plugin that reads Arduino API definitions expressed
as protobuf services and generates C++ interface headers from them.
Protobuf is used here primarily as an IDL/DSL: the first-class source of
truth is the .proto definition, while the current generated target is
a minimal C++ compatibility header.

Motivation

The goal is not to dead-copy the existing Arduino API implementation.
Instead, this establishes an alternative API definition layer that can
describe Arduino-compatible interfaces in a structured form and generate
the required C++ surface from that definition.

Although this PR currently focuses only on generating replacement/alternative
C++ headers, the same IDL model can later be used as a base for additional
generated artifacts, including RPC stubs or adapters for other protocols.

What This Adds

  • A protoc-gen-arduinoif plugin for generating C++ interface headers.
  • Protobuf custom options for C++ naming, return types, argument types,
    visibility, includes, and service inheritance.
  • IDL support for Arduino-style inheritance such as
    Print -> Stream -> HardwareSerial.
  • Generated C++ interfaces using virtual inheritance where needed.
  • A compact generation pipeline focused on the current PoC target:
    header generation only.

Design Notes

  • Protobuf services represent Arduino API classes/interfaces.
  • RPC methods represent callable API methods.
  • Request message fields represent C++ method parameters.
  • Method options map protobuf method names to Arduino-compatible C++ names,
    including overloads.
  • The protobuf definitions remain useful beyond C++ header generation because
    they preserve API semantics in an inspectable IDL form.

Current Scope

This PR intentionally keeps the implementation small.
This is just only able to build hello_arduino example at this time.
It does not introduce runtime RPC transport, client/server stubs,
or full Arduino API coverage yet.
The immediate purpose is to prove that Arduino-compatible interface headers
can be generated from the IDL cleanly.

Validation

  • Generated headers for the current Arduino API subset.
  • Verified Print, Stream, and HardwareSerial interface generation.
  • Confirmed duplicate C++ declarations are detected during generation.

@soburi soburi marked this pull request as ready for review April 29, 2026 12:19
Copilot AI review requested due to automatic review settings April 29, 2026 12:19
@soburi soburi marked this pull request as draft April 29, 2026 12:20
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces an alternative Arduino Core API definition layer for Zephyr using protobuf as an IDL, adds a protoc plugin to generate C++ interface headers from .proto services, and wires it into the build (alongside a small Rust staticlib + C++ glue for a few core functions).

Changes:

  • Add protoc-gen-arduinoif (Python) + .proto options and IDL sources, generating *_interface.hpp and *_types.h headers at build time.
  • Add alt_core_api build integration (CMake/Kconfig) and a small no_std Rust static library with C++ wrappers.
  • Adjust existing core build files/docs to point at the new IDL/codegen flow and API surface.

Reviewed changes

Copilot reviewed 32 out of 36 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
documentation/idl_codegen.md Documents the IDL/codegen pipeline and mapping rules.
cores/arduino/zephyrPrint.h Removes old Zephyr Print inline header (superseded by alt_core_api Print).
cores/arduino/zephyrCommon.cpp Minor type-cast fix around min() usage.
cores/arduino/CMakeLists.txt Removes Rust-impl-specific include/source handling (moved to alt_core_api).
alt_core_api/tools/protoc_gen_arduinoif/templates/ifc_header.hpp.j2 Jinja template for generated C++ interface headers.
alt_core_api/tools/protoc_gen_arduinoif/service_renderer.py Renders ServiceModel into generated files via Jinja.
alt_core_api/tools/protoc_gen_arduinoif/service_model.py Builds template-ready service/interface models from descriptors.
alt_core_api/tools/protoc_gen_arduinoif/request_context.py Indexes messages/services for a CodeGeneratorRequest.
alt_core_api/tools/protoc_gen_arduinoif/method_spec.py Derives C++ method signatures from RPC/input/output descriptors.
alt_core_api/tools/protoc_gen_arduinoif/enum_renderer.py Emits C-compatible enum-only headers for shared proto files.
alt_core_api/tools/protoc_gen_arduinoif/core.py Plugin entry logic: selects files, renders enums/services, returns response.
alt_core_api/tools/protoc_gen_arduinoif/common.py Shared helpers + dynamic loading of generated arduino_opts_pb2.py.
alt_core_api/tools/protoc-gen-arduinoif Protoc plugin executable wrapper.
alt_core_api/rust/src/lib.rs no_std Rust crate root + panic handler.
alt_core_api/rust/src/common.rs Rust implementations for a few Arduino-like core helpers (map/makeWord).
alt_core_api/rust/Cargo.toml Rust crate metadata for building a staticlib.
alt_core_api/idl/proto/print.proto IDL for Print interface (methods/options).
alt_core_api/idl/proto/stream.proto IDL for Stream interface with inheritance from Print.
alt_core_api/idl/proto/hardware_serial.proto IDL for HardwareSerial interface + related enums.
alt_core_api/idl/proto/common.proto Shared enums intended to generate common_types.h.
alt_core_api/idl/proto/arduino_opts.proto Custom protobuf options used by the generator.
alt_core_api/idl/proto/google/protobuf/wrappers.proto Vendored well-known type used by the IDL (wrappers).
alt_core_api/idl/proto/google/protobuf/empty.proto Vendored well-known type used by the IDL (Empty).
alt_core_api/idl/CMakeLists.txt CMake pipeline to generate pb2 options + generated interface/type headers.
alt_core_api/api/apiPrint.cpp Keeps Zephyr print helpers; removes default Print::write(buffer,size) body.
alt_core_api/api/apiCommon.cpp C++ wrappers calling Rust extern "C" helpers.
alt_core_api/api/Print.h New Print API surface based on generated PrintInterface.
alt_core_api/api/Stream.h New Stream API surface based on generated StreamInterface.
alt_core_api/api/HardwareSerial.h New HardwareSerial API surface based on generated interface header.
alt_core_api/api/Common.h Stub compatibility header.
alt_core_api/api/ArduinoAPI.h Minimal Arduino API compatibility header (types/min/setup/loop).
alt_core_api/api/CMakeLists.txt Adds alt_core_api C++ sources to the build.
alt_core_api/CMakeLists.txt Builds Rust staticlib, runs IDL generation, and adds alt_core_api subdirs.
README.md Links the new IDL/codegen documentation.
Kconfig Selects NANOPB when Rust implementation is enabled.
CMakeLists.txt Switches Rust-impl branch to include/build alt_core_api.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread alt_core_api/api/Print.h Outdated
Comment thread alt_core_api/api/HardwareSerial.h
Comment thread alt_core_api/tools/protoc_gen_arduinoif/service_model.py Outdated
Comment thread alt_core_api/api/Stream.h Outdated
Comment thread alt_core_api/api/Print.h Outdated
@soburi soburi force-pushed the alt-api branch 2 times, most recently from 147a1c4 to a4d1ca9 Compare April 29, 2026 13:25
@soburi soburi requested a review from Copilot April 29, 2026 13:26
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces an IDL-driven (protobuf) definition of an Arduino-compatible API surface and a protoc plugin (protoc-gen-arduinoif) that generates minimal C++ interface headers from .proto service definitions, integrating the generation step into the Zephyr/CMake build for alt_core_api.

Changes:

  • Added a Python-based protoc plugin and templates to render C++ pure-virtual interface headers (plus enum-only C headers) from protobuf IDL.
  • Added Arduino API IDL .proto files (+ custom options) and CMake glue to generate headers into the build tree and add them to include paths.
  • Added an alt_core_api compatibility layer (C++ headers + small API shims + Rust staticlib build) and wired it into the build when CONFIG_USE_ARDUINO_API_RUST_IMPLEMENTATION is enabled.

Reviewed changes

Copilot reviewed 32 out of 37 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
documentation/idl_codegen.md New documentation describing the IDL/codegen flow and validation rules.
cores/arduino/zephyrPrint.h Removes old Zephyr-specific Print inline helpers header.
cores/arduino/zephyrCommon.cpp Minor formatting/type-cast adjustment for tone() counter computation.
cores/arduino/CMakeLists.txt Removes Rust-implementation-specific include/source conditional blocks.
alt_core_api/tools/protoc_gen_arduinoif/templates/ifc_header.hpp.j2 Jinja2 template for generated C++ interface headers.
alt_core_api/tools/protoc_gen_arduinoif/service_renderer.py Jinja2-based renderer for service/interface headers.
alt_core_api/tools/protoc_gen_arduinoif/service_model.py Builds a template-ready model from service descriptors/options (incl. inheritance).
alt_core_api/tools/protoc_gen_arduinoif/request_context.py Builds request-wide indexes for messages/services and caches inheritance lineage.
alt_core_api/tools/protoc_gen_arduinoif/method_spec.py Converts RPC descriptors into C++ method declarations (types/names/visibility).
alt_core_api/tools/protoc_gen_arduinoif/enum_renderer.py Renders enum-only proto files into C-compatible headers.
alt_core_api/tools/protoc_gen_arduinoif/core.py Plugin entry logic: reads CodeGeneratorRequest, emits generated files.
alt_core_api/tools/protoc_gen_arduinoif/common.py Shared helpers (naming, uniq) + dynamic load of arduino_opts_pb2.py via env var.
alt_core_api/tools/protoc-gen-arduinoif Executable plugin wrapper that sets sys.path and dispatches to core.main().
alt_core_api/rust/src/lib.rs no_std Rust staticlib crate root + panic handler.
alt_core_api/rust/src/common.rs Rust implementations exported for C/C++ linkage (map_i32, makeWord_*).
alt_core_api/rust/Cargo.toml Rust crate metadata for producing a staticlib.
alt_core_api/idl/proto/stream.proto IDL for Stream service (inherits Print) + LookaheadMode enum.
alt_core_api/idl/proto/print.proto IDL for Print service and request messages for write overloads.
alt_core_api/idl/proto/hardware_serial.proto IDL for HardwareSerial service (inherits Stream) + serial config enums.
alt_core_api/idl/proto/google/protobuf/wrappers.proto Vendored well-known-type proto used for scalar return wrappers.
alt_core_api/idl/proto/google/protobuf/empty.proto Vendored google.protobuf.Empty proto used for void/no-arg mapping.
alt_core_api/idl/proto/common.proto Shared enum-only IDL intended to generate *_types.h compatibility header.
alt_core_api/idl/proto/arduino_opts.proto Custom protobuf options/extensions for C++ name/type/visibility/inheritance/includes.
alt_core_api/idl/CMakeLists.txt CMake pipeline: generate arduino_opts_pb2.py, run plugin, add include dirs/targets.
alt_core_api/api/apiPrint.cpp Removes Print::write(buffer,size) body (now provided inline in new header).
alt_core_api/api/apiCommon.cpp Adds C++ shims calling Rust exports for map() and makeWord().
alt_core_api/api/Stream.h Adds Stream compatibility header inheriting from generated StreamInterface.
alt_core_api/api/Print.h Adds Print compatibility header inheriting from generated PrintInterface + inline helpers.
alt_core_api/api/HardwareSerial.h Adds HardwareSerial compatibility header inheriting generated HardwareSerialInterface.
alt_core_api/api/Common.h Adds a stub Common.h for compatibility.
alt_core_api/api/CMakeLists.txt Adds apiCommon.cpp and apiPrint.cpp to the build.
alt_core_api/api/ArduinoAPI.h Adds minimal Arduino core type/function declarations (e.g., byte, pin_size_t, min).
alt_core_api/CMakeLists.txt Adds Rust staticlib build + links it; adds idl/ and api/ subdirs and dependencies.
README.md Adds a link to the new IDL/codegen documentation.
Kconfig Ensures NANOPB is selected when enabling the Rust implementation path.
CMakeLists.txt Switches the Rust-implementation build path to include alt_core_api instead of the blobbed API.
.gitignore Updates ignored Rust lockfile path to alt_core_api/rust/Cargo.lock.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread alt_core_api/tools/protoc_gen_arduinoif/method_spec.py
Comment thread alt_core_api/tools/protoc_gen_arduinoif/request_context.py Outdated
Comment thread alt_core_api/tools/protoc_gen_arduinoif/method_spec.py Outdated
Comment thread alt_core_api/api/Print.h Outdated
Comment thread documentation/idl_codegen.md
@soburi soburi force-pushed the alt-api branch 2 times, most recently from 12b0f14 to 5670074 Compare April 30, 2026 22:44
@soburi soburi requested a review from Copilot April 30, 2026 22:45
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces an IDL-driven code generation pipeline (protoc-gen-arduinoif) for producing Arduino-compatible C++ interface headers from .proto service definitions, and wires it into the Zephyr build as an alternative “core API” implementation path.

Changes:

  • Added a Python protoc plugin + templates to generate C++ interface headers (and enum-only type headers) from Arduino API .proto IDL.
  • Added initial Arduino IDL protos (Print, Stream, HardwareSerial, plus shared enums/options) and CMake integration to generate headers during builds.
  • Added an alt_core_api implementation path (C++ headers + some Rust-backed C ABI helpers) and hooked it up behind CONFIG_USE_ARDUINO_API_RUST_IMPLEMENTATION.

Reviewed changes

Copilot reviewed 32 out of 37 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
documentation/idl_codegen.md Documents the new Arduino IDL/codegen flow and build integration.
cores/arduino/zephyrPrint.h Removes the previous Zephyr Print inline helper header.
cores/arduino/zephyrCommon.cpp Formatting / type adjustment for min() usage in tone().
cores/arduino/CMakeLists.txt Removes rust-specific include/source tweaks in this directory.
alt_core_api/tools/protoc_gen_arduinoif/templates/ifc_header.hpp.j2 Jinja template for generated C++ interface headers.
alt_core_api/tools/protoc_gen_arduinoif/service_renderer.py Renders ServiceModel via Jinja templates.
alt_core_api/tools/protoc_gen_arduinoif/service_model.py Builds template-ready service models incl. inheritance/includes.
alt_core_api/tools/protoc_gen_arduinoif/request_context.py Builds request-wide indices for messages/services/lineage cache.
alt_core_api/tools/protoc_gen_arduinoif/method_spec.py Maps RPCs and message fields into C++ method signatures.
alt_core_api/tools/protoc_gen_arduinoif/enum_renderer.py Emits enum-only C headers for shared proto files.
alt_core_api/tools/protoc_gen_arduinoif/core.py Plugin entry logic: parse request and emit generated files.
alt_core_api/tools/protoc_gen_arduinoif/common.py Shared helpers + dynamic loading of arduino_opts_pb2.
alt_core_api/tools/protoc-gen-arduinoif protoc plugin entrypoint script.
alt_core_api/rust/src/lib.rs no_std Rust library scaffolding + panic handler.
alt_core_api/rust/src/common.rs Exports C ABI helpers (e.g., map_i32, makeWord_*).
alt_core_api/rust/Cargo.toml Rust staticlib crate definition for alt_core_api.
alt_core_api/idl/proto/stream.proto IDL proto for Stream and its methods + inheritance from Print.
alt_core_api/idl/proto/print.proto IDL proto for Print incl. overload mapping to write.
alt_core_api/idl/proto/hardware_serial.proto IDL proto for HardwareSerial + enums/config and inheritance from Stream.
alt_core_api/idl/proto/google/protobuf/wrappers.proto Vendored wrappers proto used by the IDL.
alt_core_api/idl/proto/google/protobuf/empty.proto Vendored Empty proto used by the IDL.
alt_core_api/idl/proto/common.proto Shared enum-only proto intended to generate *_types.h.
alt_core_api/idl/proto/arduino_opts.proto Custom options for C++ names/types/includes/inheritance/visibility.
alt_core_api/idl/CMakeLists.txt CMake: generate arduino_opts_pb2.py + run plugin to produce headers.
alt_core_api/api/apiPrint.cpp Removes Print::write(buffer,size) impl (moved inline to header).
alt_core_api/api/apiCommon.cpp New C++ wrappers that call into Rust-exported C functions.
alt_core_api/api/Stream.h C++ API header bridging Stream to generated StreamInterface.
alt_core_api/api/Print.h C++ API header bridging Print to generated PrintInterface.
alt_core_api/api/HardwareSerial.h C++ API header bridging HardwareSerial to generated interface + enums.
alt_core_api/api/Common.h Stub compatibility header.
alt_core_api/api/CMakeLists.txt Adds apiCommon.cpp and apiPrint.cpp to the build.
alt_core_api/api/ArduinoAPI.h Arduino API surface/types for the alt_core_api path.
alt_core_api/CMakeLists.txt Builds Rust staticlib, adds IDL + API subdirs, wires dependencies.
README.md Adds link to the new IDL/codegen documentation.
Kconfig Ensures NANOPB is selected when using the Rust-based implementation.
CMakeLists.txt Switches Rust implementation path to build alt_core_api instead of inlined Rust build logic.
.gitignore Updates ignored Cargo.lock location to alt_core_api/rust/Cargo.lock.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread alt_core_api/idl/CMakeLists.txt
Comment thread alt_core_api/idl/CMakeLists.txt Outdated
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces an Arduino API IDL layer (protobuf-based) and a protoc plugin to generate Arduino-compatible C++ interface headers, then wires that generated surface into an alternative alt_core_api build path (including a small Rust staticlib).

Changes:

  • Added protoc-gen-arduinoif (Python) plus templates/models to generate C++ pure-virtual interface headers from .proto services and enums.
  • Added initial Arduino IDL .proto definitions (e.g., Print, Stream, HardwareSerial, and shared enums) and CMake integration to run the generator during builds.
  • Added alt_core_api compatibility headers/impls and a Rust staticlib, and switched the “Rust implementation” build path to build alt_core_api.

Reviewed changes

Copilot reviewed 31 out of 36 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
documentation/idl_codegen.md Documents the IDL/codegen design, mapping rules, and build integration.
cores/arduino/zephyrPrint.h Removes the old Zephyr-specific Print inline header implementation.
cores/arduino/zephyrCommon.cpp Adjusts min() call to avoid mixed-type template deduction issues.
cores/arduino/CMakeLists.txt Removes Rust-mode conditional sources/include ordering logic.
alt_core_api/tools/protoc_gen_arduinoif/templates/ifc_header.hpp.j2 Jinja2 template for generated C++ interface headers (methods, enums, inheritance).
alt_core_api/tools/protoc_gen_arduinoif/service_renderer.py Renders ServiceModel using the interface-header template.
alt_core_api/tools/protoc_gen_arduinoif/service_model.py Builds template-ready service/interface models incl. inheritance and includes.
alt_core_api/tools/protoc_gen_arduinoif/request_context.py Builds per-request indexes for messages/services used by the generator.
alt_core_api/tools/protoc_gen_arduinoif/method_spec.py Converts RPC methods + options into C++ method declarations/keys.
alt_core_api/tools/protoc_gen_arduinoif/enum_renderer.py Generates _types.h headers for enum-only proto files.
alt_core_api/tools/protoc_gen_arduinoif/core.py protoc plugin core: parses request and emits generated files.
alt_core_api/tools/protoc_gen_arduinoif/common.py Shared helpers (naming, uniq) and dynamic loading of opts pb2 module.
alt_core_api/tools/protoc-gen-arduinoif Plugin entrypoint script invoked by protoc.
alt_core_api/rust/src/lib.rs no_std Rust library scaffold and panic handler for the staticlib.
alt_core_api/rust/src/common.rs Rust exports for core Arduino-like helpers (map, makeWord).
alt_core_api/rust/Cargo.toml Rust crate configuration (staticlib).
alt_core_api/idl/proto/stream.proto IDL for Stream service and LookaheadMode enum.
alt_core_api/idl/proto/print.proto IDL for Print service + write/flush/availableForWrite mapping.
alt_core_api/idl/proto/hardware_serial.proto IDL for HardwareSerial service plus serial-related enums.
alt_core_api/idl/proto/google/protobuf/wrappers.proto Vendored protobuf wrappers used as IDL return message types.
alt_core_api/idl/proto/google/protobuf/empty.proto Vendored protobuf Empty message for void/no-args.
alt_core_api/idl/proto/common.proto Shared Arduino enums emitted as *_types.h.
alt_core_api/idl/proto/arduino_opts.proto Custom protobuf options for C++ codegen behavior.
alt_core_api/idl/CMakeLists.txt Adds nanopb + codegen pipeline to generate interface headers during build.
alt_core_api/api/apiPrint.cpp Keeps Zephyr print helpers; removes out-of-line default Print::write(buffer,size).
alt_core_api/api/apiCommon.cpp Adds C++ wrappers calling into Rust exports for common Arduino helpers.
alt_core_api/api/Stream.h Defines arduino::Stream in terms of generated StreamInterface.
alt_core_api/api/Print.h Defines arduino::Print in terms of generated PrintInterface + inline helpers.
alt_core_api/api/HardwareSerial.h Defines arduino::HardwareSerial in terms of generated interface + config enum.
alt_core_api/api/Common.h Adds a compatibility stub header.
alt_core_api/api/CMakeLists.txt Builds the alt_core_api C++ API sources.
alt_core_api/api/ArduinoAPI.h Minimal Arduino API base types/utilities for the alt API path.
alt_core_api/CMakeLists.txt Builds the Rust staticlib and adds idl/ + api/ subdirectories.
README.md Links new IDL codegen documentation.
CMakeLists.txt Switches the Rust-implementation path to include/build alt_core_api.
.gitignore Updates ignored Cargo.lock path to alt_core_api/rust/Cargo.lock.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread alt_core_api/idl/CMakeLists.txt
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces an Arduino API IDL layer (protobuf-based) plus a protoc plugin that generates Arduino-compatible C++ interface headers, and wires that into the Zephyr build via the new alt_core_api module (including an initial Rust staticlib).

Changes:

  • Add protoc-gen-arduinoif (Python) and protobuf option schema to generate C++ interface headers from .proto services.
  • Add initial Arduino IDL .proto sources (Print/Stream/HardwareSerial + shared enums) and CMake integration to generate headers during builds.
  • Add alt_core_api compatibility headers and a Rust static library build/link step, updating top-level build selection and docs.

Reviewed changes

Copilot reviewed 31 out of 36 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
documentation/idl_codegen.md Documents the IDL/codegen pipeline and how to invoke it.
cores/arduino/zephyrPrint.h Removes the old Zephyr Print inline header.
cores/arduino/zephyrCommon.cpp Adjusts min() call typing/formatting for tone counter computation.
cores/arduino/CMakeLists.txt Removes Rust-implementation conditional include/source wiring from core.
alt_core_api/tools/protoc_gen_arduinoif/templates/ifc_header.hpp.j2 Jinja template for generated C++ interface headers.
alt_core_api/tools/protoc_gen_arduinoif/service_renderer.py Renders ServiceModel into generated interface header files.
alt_core_api/tools/protoc_gen_arduinoif/service_model.py Builds the template-ready model (includes, inheritance, methods, enums).
alt_core_api/tools/protoc_gen_arduinoif/request_context.py Indexes messages/services across the CodeGeneratorRequest.
alt_core_api/tools/protoc_gen_arduinoif/method_spec.py Converts RPC + request/response messages into C++ method signatures.
alt_core_api/tools/protoc_gen_arduinoif/enum_renderer.py Emits C-compatible headers for enum-only proto files.
alt_core_api/tools/protoc_gen_arduinoif/core.py Main protoc plugin implementation and error handling.
alt_core_api/tools/protoc_gen_arduinoif/common.py Shared helpers + dynamic loading of arduino_opts_pb2.py.
alt_core_api/tools/protoc-gen-arduinoif Plugin entrypoint script used by protoc.
alt_core_api/rust/src/lib.rs Defines the no_std Rust crate entry and panic handler.
alt_core_api/rust/src/common.rs Implements C ABI helpers in Rust (e.g., map_i32, makeWord_*).
alt_core_api/rust/Cargo.toml Rust crate definition for building a staticlib.
alt_core_api/idl/proto/stream.proto Defines Stream service and LookaheadMode enum in IDL.
alt_core_api/idl/proto/print.proto Defines Print service and write/flush APIs in IDL.
alt_core_api/idl/proto/hardware_serial.proto Defines HardwareSerial service and serial config enums in IDL.
alt_core_api/idl/proto/google/protobuf/wrappers.proto Vendors protobuf wrappers proto for local IDL compilation.
alt_core_api/idl/proto/google/protobuf/empty.proto Vendors protobuf Empty proto for local IDL compilation.
alt_core_api/idl/proto/common.proto Shared Arduino enums intended for _types.h generation.
alt_core_api/idl/proto/arduino_opts.proto Declares custom protobuf options for C++ generation control.
alt_core_api/idl/CMakeLists.txt Adds nanopb integration + custom commands/targets to run codegen.
alt_core_api/api/apiPrint.cpp Keeps Zephyr Print helpers; removes moved default Print::write impl.
alt_core_api/api/apiCommon.cpp Adds C++ wrappers calling into Rust ABI implementations.
alt_core_api/api/Stream.h Adds Stream compatibility type inheriting generated interface.
alt_core_api/api/Print.h Adds Print compatibility type inheriting generated interface + helpers.
alt_core_api/api/HardwareSerial.h Adds HardwareSerial compatibility type inheriting generated interface.
alt_core_api/api/Common.h Adds a stub Common header for compatibility.
alt_core_api/api/CMakeLists.txt Adds alt_core_api C++ sources to the build.
alt_core_api/api/ArduinoAPI.h Adds the minimal Arduino API surface/types used by cores.
alt_core_api/CMakeLists.txt Adds Rust build/link + hooks in IDL and API subdirectories.
README.md Links to the new IDL/codegen documentation.
CMakeLists.txt Switches Rust-implementation branch to include/build alt_core_api.
.gitignore Updates ignored Cargo.lock location for the new Rust crate path.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread alt_core_api/idl/CMakeLists.txt
Comment thread alt_core_api/idl/CMakeLists.txt Outdated
Comment thread alt_core_api/api/Print.h Outdated
Comment thread alt_core_api/api/Print.h Outdated
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a protobuf-based IDL for Arduino-compatible APIs and a protoc plugin (protoc-gen-arduinoif) that generates minimal C++ interface headers from .proto service definitions, integrating this into the Zephyr/CMake build and wiring an alternative alt_core_api implementation path (including a Rust staticlib).

Changes:

  • Added a Python protoc plugin + templates to render C++ interface headers (and enum-only C type headers) from .proto services/enums.
  • Added initial Arduino IDL .proto definitions (Print, Stream, HardwareSerial, shared common enums) and CMake integration to generate headers during the build.
  • Added alt_core_api headers/sources (and Rust staticlib build) to provide an alternative Arduino API surface when CONFIG_USE_ARDUINO_API_RUST_IMPLEMENTATION is enabled.

Reviewed changes

Copilot reviewed 32 out of 37 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
documentation/idl_codegen.md Documents the IDL/codegen flow, mappings, and build integration.
cores/arduino/zephyrPrint.h Removes the previous Zephyr-specific Print inline header implementation.
cores/arduino/zephyrCommon.cpp Minor update to tone() count computation formatting/casting.
cores/arduino/CMakeLists.txt Removes conditional rust-impl include/source wiring (moved to alt_core_api).
alt_core_api/tools/protoc_gen_arduinoif/templates/ifc_header.hpp.j2 Jinja template for generated C++ interface headers.
alt_core_api/tools/protoc_gen_arduinoif/service_renderer.py Renders ServiceModel into generated files via Jinja.
alt_core_api/tools/protoc_gen_arduinoif/service_model.py Builds template-ready service models (includes inheritance/includes/overload checks).
alt_core_api/tools/protoc_gen_arduinoif/request_context.py Indexes request descriptors (messages/services) for generation.
alt_core_api/tools/protoc_gen_arduinoif/method_spec.py Maps RPCs + message fields/options into C++ method signatures.
alt_core_api/tools/protoc_gen_arduinoif/enum_renderer.py Emits C-compatible headers for enum-only proto files.
alt_core_api/tools/protoc_gen_arduinoif/core.py Plugin entry logic: parse request and emit generated files or errors.
alt_core_api/tools/protoc_gen_arduinoif/common.py Shared helpers + dynamic loading of arduino_opts_pb2.py via env var.
alt_core_api/tools/protoc-gen-arduinoif Executable plugin entrypoint script for protoc.
alt_core_api/rust/src/lib.rs no_std Rust crate entry with panic handler for staticlib.
alt_core_api/rust/src/common.rs Rust FFI functions intended to back Arduino API helpers (map, makeWord).
alt_core_api/rust/Cargo.toml Declares Rust staticlib crate configuration.
alt_core_api/idl/proto/stream.proto Defines Stream service + enums and inheritance from Print.
alt_core_api/idl/proto/print.proto Defines Print service and method mapping options (overloads via cpp_name).
alt_core_api/idl/proto/hardware_serial.proto Defines HardwareSerial service + enums and inheritance from Stream.
alt_core_api/idl/proto/google/protobuf/wrappers.proto Vendored well-known type definitions used by the IDL.
alt_core_api/idl/proto/google/protobuf/empty.proto Vendored google.protobuf.Empty definition used by the IDL.
alt_core_api/idl/proto/common.proto Shared Arduino-ish enum definitions for type header generation.
alt_core_api/idl/proto/arduino_opts.proto Protobuf custom options used to control C++ generation.
alt_core_api/idl/CMakeLists.txt CMake pipeline for generating pb2 options + generated interface/type headers.
alt_core_api/api/apiPrint.cpp Keeps Zephyr printing helpers; removes the Print::write(buffer,size) impl here.
alt_core_api/api/apiCommon.cpp Adds C++ wrappers calling into Rust FFI (map_i32, makeWord_*).
alt_core_api/api/Stream.h Defines Stream C++ type inheriting generated StreamInterface.
alt_core_api/api/Print.h Defines Print type inheriting generated PrintInterface + inline helpers/defaults.
alt_core_api/api/HardwareSerial.h Defines HardwareSerial type inheriting generated HardwareSerialInterface.
alt_core_api/api/Common.h Adds stub compatibility header.
alt_core_api/api/CMakeLists.txt Adds alt_core_api/api sources to the build.
alt_core_api/api/ArduinoAPI.h Defines core Arduino API types/helpers for the alt_core_api path.
alt_core_api/CMakeLists.txt Adds Rust build + links staticlib; adds idl and api subdirectories.
README.md Adds link to the new IDL codegen documentation.
CMakeLists.txt Switches rust-implementation branch to build alt_core_api instead of blob sources.
.gitignore Updates ignored Cargo.lock path to the new rust crate location.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread alt_core_api/api/ArduinoAPI.h Outdated
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Introduces a protobuf-based IDL layer and a protoc plugin (protoc-gen-arduinoif) to generate Arduino-compatible C++ interface headers, and wires this into the Zephyr/CMake build when using the alternative (Rust-backed) API implementation.

Changes:

  • Added a Python protoc plugin + Jinja2 templates to generate *_interface.hpp and *_types.h headers from .proto service/enum definitions.
  • Added Arduino IDL .proto sources (including custom options) and CMake build steps to generate headers during builds.
  • Added an alternative API surface (alt_core_api/api) plus Rust staticlib build integration, and updated top-level build wiring/docs accordingly.

Reviewed changes

Copilot reviewed 31 out of 36 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
documentation/idl_codegen.md Documents the IDL/codegen flow, build integration, and validation rules.
cores/arduino/zephyrPrint.h Removes legacy inline Print helpers previously located under cores/arduino.
cores/arduino/CMakeLists.txt Removes Rust-implementation-specific include/source wiring from the core build.
alt_core_api/tools/protoc_gen_arduinoif/templates/ifc_header.hpp.j2 Adds the Jinja template for generated C++ interface headers.
alt_core_api/tools/protoc_gen_arduinoif/service_renderer.py Adds renderer that produces interface headers from the service model.
alt_core_api/tools/protoc_gen_arduinoif/service_model.py Adds service-to-model builder including inheritance/include handling.
alt_core_api/tools/protoc_gen_arduinoif/request_context.py Adds request indexing for messages/services and lineage cache.
alt_core_api/tools/protoc_gen_arduinoif/method_spec.py Adds RPC-to-C++ method signature inference and option handling.
alt_core_api/tools/protoc_gen_arduinoif/enum_renderer.py Adds generation of enum-only *_types.h headers.
alt_core_api/tools/protoc_gen_arduinoif/core.py Adds plugin entry logic for iterating generated outputs.
alt_core_api/tools/protoc_gen_arduinoif/common.py Adds shared helpers and dynamic loading of arduino_opts_pb2.py.
alt_core_api/tools/protoc-gen-arduinoif Adds the protoc plugin entrypoint script.
alt_core_api/rust/src/lib.rs Adds no_std Rust crate root and panic handler.
alt_core_api/rust/src/common.rs Adds Rust-backed implementations intended to be called from C/C++.
alt_core_api/rust/Cargo.toml Adds Rust crate metadata for building a static library.
alt_core_api/idl/proto/print.proto Defines Print service/methods in the IDL.
alt_core_api/idl/proto/stream.proto Defines Stream service and inheritance from Print in the IDL.
alt_core_api/idl/proto/hardware_serial.proto Defines HardwareSerial service, enums, and inheritance from Stream.
alt_core_api/idl/proto/common.proto Defines shared enums to be emitted as *_types.h.
alt_core_api/idl/proto/arduino_opts.proto Defines custom protobuf options driving C++ codegen behavior.
alt_core_api/idl/proto/google/protobuf/wrappers.proto Vendors well-known wrappers needed by the IDL.
alt_core_api/idl/proto/google/protobuf/empty.proto Vendors google.protobuf.Empty needed by the IDL.
alt_core_api/idl/proto/google/protobuf/descriptor.proto Vendors descriptor proto to support custom options.
alt_core_api/idl/CMakeLists.txt Adds CMake pipeline to generate headers via the plugin and include them in builds.
alt_core_api/api/apiPrint.cpp Keeps Zephyr Print formatting helpers; removes duplicated default write() impl.
alt_core_api/api/apiCommon.cpp Adds C/C++ shims that call into the Rust staticlib.
alt_core_api/api/Print.h Adds Print class wired to generated PrintInterface plus inline helpers/defaults.
alt_core_api/api/Stream.h Adds Stream class wired to generated StreamInterface.
alt_core_api/api/HardwareSerial.h Adds HardwareSerial class wired to generated HardwareSerialInterface.
alt_core_api/api/Common.h Adds a stub compatibility header.
alt_core_api/api/ArduinoAPI.h Adds Arduino API header surface (types/funcs) for the alt core.
alt_core_api/api/CMakeLists.txt Adds build sources for the alt API C++ layer.
alt_core_api/CMakeLists.txt Adds Rust staticlib build/link integration + includes the IDL and API subdirs.
README.md Links the new IDL/codegen documentation.
CMakeLists.txt Switches Rust-implementation builds to include alt_core_api instead of the blob API.
.gitignore Adjusts ignored Cargo.lock path to match new Rust crate location.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread alt_core_api/tools/protoc_gen_arduinoif/method_spec.py Outdated
Comment thread alt_core_api/api/ArduinoAPI.h Outdated
@soburi soburi force-pushed the alt-api branch 3 times, most recently from c690b97 to 4466487 Compare May 1, 2026 02:29
@soburi soburi requested a review from Copilot May 1, 2026 02:29
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces an IDL/code-generation pipeline (protoc-gen-arduinoif) that treats .proto service definitions as the source-of-truth for Arduino-compatible APIs, generating C++ “interface header” surfaces and wiring that into the alt_core_api build when the Rust implementation is enabled.

Changes:

  • Add a Python protoc plugin + templates to generate C++ interface headers from protobuf service descriptors (with custom options for C++ naming/types/inheritance).
  • Add initial Arduino IDL protos (Print, Stream, HardwareSerial, and shared enums) and CMake integration to generate headers at build time.
  • Introduce alt_core_api C++ compatibility headers and a Rust staticlib (with C++ wrappers) used when CONFIG_USE_ARDUINO_API_RUST_IMPLEMENTATION is enabled.

Reviewed changes

Copilot reviewed 33 out of 38 changed files in this pull request and generated no comments.

Show a summary per file
File Description
documentation/idl_codegen.md Adds documentation for the protoc plugin flow, build integration, and IDL mapping rules.
cores/arduino/zephyrPrint.h Removes the old Zephyr-specific Print inline implementation header.
cores/arduino/zephyrCommon.cpp Adjusts min() usage to avoid truncation/dangling issues in tone() count calculation.
cores/arduino/CMakeLists.txt Removes Rust-implementation-specific include/source conditionals now handled via alt_core_api.
alt_core_api/tools/protoc_gen_arduinoif/templates/ifc_header.hpp.j2 Adds Jinja template for generated C++ interface headers.
alt_core_api/tools/protoc_gen_arduinoif/service_renderer.py Adds renderer that turns ServiceModel into generated files via Jinja.
alt_core_api/tools/protoc_gen_arduinoif/service_model.py Adds model builder with inheritance resolution, include collection, and overload validation.
alt_core_api/tools/protoc_gen_arduinoif/request_context.py Adds request-wide descriptor indexes for messages/services and lineage caching.
alt_core_api/tools/protoc_gen_arduinoif/method_spec.py Adds method signature derivation from RPCs + options, with validation.
alt_core_api/tools/protoc_gen_arduinoif/enum_renderer.py Adds enum-only proto rendering to *_types.h.
alt_core_api/tools/protoc_gen_arduinoif/core.py Adds plugin entry logic: parse request, generate headers, report errors.
alt_core_api/tools/protoc_gen_arduinoif/common.py Adds shared utilities and PROTOC_GEN_ARDUINOIF_PB2 loader for custom options.
alt_core_api/tools/protoc-gen-arduinoif Adds the protoc plugin executable entrypoint script.
alt_core_api/rust/src/lib.rs Adds no_std Rust crate entry and panic handler for staticlib use.
alt_core_api/rust/src/common.rs Adds Rust FFI implementations for core helpers (map, makeWord).
alt_core_api/rust/Cargo.toml Adds Rust crate metadata for building a staticlib.
alt_core_api/idl/proto/print.proto Defines Print service and request messages, mapping to C++ write overloads and helpers.
alt_core_api/idl/proto/stream.proto Defines Stream service inheriting from Print plus LookaheadMode enum.
alt_core_api/idl/proto/hardware_serial.proto Defines HardwareSerial service inheriting from Stream plus serial config enums and operator bool.
alt_core_api/idl/proto/common.proto Adds shared Arduino enums for generated common_types.h.
alt_core_api/idl/proto/arduino_opts.proto Adds custom protobuf options used to control C++ generation.
alt_core_api/idl/proto/google/protobuf/empty.proto Vendors protobuf well-known type for Empty.
alt_core_api/idl/proto/google/protobuf/wrappers.proto Vendors protobuf well-known wrapper messages for scalar returns.
alt_core_api/idl/proto/google/protobuf/descriptor.proto Vendors descriptor.proto to support custom options compilation.
alt_core_api/idl/CMakeLists.txt Adds CMake pipeline to generate arduino_opts_pb2.py then run protoc-gen-arduinoif.
alt_core_api/api/apiPrint.cpp Removes Print::write(const uint8_t*, size_t) implementation from .cpp (now inline in header).
alt_core_api/api/apiCommon.cpp Adds C++ wrappers that call into Rust FFI for map()/makeWord().
alt_core_api/api/Print.h Adds compatibility Print class bridging generated PrintInterface and Zephyr print helpers.
alt_core_api/api/Stream.h Adds compatibility Stream class bridging generated StreamInterface.
alt_core_api/api/HardwareSerial.h Adds compatibility HardwareSerial class bridging generated HardwareSerialInterface.
alt_core_api/api/Common.h Adds stub Common.h for compatibility includes.
alt_core_api/api/ArduinoAPI.h Adds minimal Arduino API surface/types and includes generated shared enum header.
alt_core_api/api/CMakeLists.txt Adds Zephyr build sources for the compatibility API layer.
alt_core_api/CMakeLists.txt Adds Rust build (cargo) + links staticlib and integrates IDL + API subdirs.
README.md Links the new Arduino IDL code generation documentation.
Kconfig Ensures NANOPB is selected when using the Rust implementation path.
CMakeLists.txt Switches Rust-implementation build to include and build alt_core_api instead of inline Rust build logic.
.gitignore Updates ignored Cargo.lock path to alt_core_api/rust/Cargo.lock.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@soburi soburi marked this pull request as ready for review May 3, 2026 14:26
soburi added 2 commits May 4, 2026 06:00
Add `protoc-gen-arduinoif`, a protobuf plugin that generates
IPC/interface headers from IDL `.proto` files.

The plugin treats protobuf services as the IDL source and emits
the current header surface used by `alt_core_api`.
Runtime IPC/RPC transport and stub implementations are intentionally
 out of scope for now.

Signed-off-by: TOKITA Hiroshi <tokita.hiroshi@gmail.com>
Summary
-------

This adds a `protoc` plugin that reads Arduino API definitions expressed
as protobuf services and generates C++ interface headers from them.
Protobuf is used here primarily as an IDL/DSL: the first-class source of
truth is the `.proto` definition, while the current generated target is
a minimal C++ compatibility header.

Motivation
----------

The goal is not to dead-copy the existing Arduino API implementation.
Instead, this establishes an alternative API definition layer that can
describe Arduino-compatible interfaces in a structured form and generate
the required C++ surface from that definition.

Although this PR currently focuses only on generating replacement/alternative
C++ headers, the same IDL model can later be used as a base for additional
generated artifacts, including RPC stubs or adapters for other protocols.

What This Adds
--------------

- A `protoc-gen-arduinoif` plugin for generating C++ interface headers.
- Protobuf custom options for C++ naming, return types, argument types,
  visibility, includes, and service inheritance.
- IDL support for Arduino-style inheritance such as
  `Print -> Stream -> HardwareSerial`.
- Generated C++ interfaces using virtual inheritance where needed.
- A compact generation pipeline focused on the current PoC target:
  header generation only.

Design Notes
------------

- Protobuf services represent Arduino API classes/interfaces.
- RPC methods represent callable API methods.
- Request message fields represent C++ method parameters.
- Method options map protobuf method names to Arduino-compatible C++ names,
  including overloads.
- The protobuf definitions remain useful beyond C++ header generation because
  they preserve API semantics in an inspectable IDL form.

Current Scope
-------------

This PR intentionally keeps the implementation small.
This is just only able to build hello_arduino example at this time.
It does not introduce runtime RPC transport, client/server stubs,
or full Arduino API coverage yet.
The immediate purpose is to prove that Arduino-compatible interface headers
can be generated from the IDL cleanly.

Validation
----------

- Generated headers for the current Arduino API subset.
- Verified `Print`, `Stream`, and `HardwareSerial` interface generation.
- Confirmed duplicate C++ declarations are detected during generation.

Signed-off-by: TOKITA Hiroshi <tokita.hiroshi@gmail.com>
@soburi soburi added the DNM This PR should not be merged (Do Not Merge) label May 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DNM This PR should not be merged (Do Not Merge)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants