docs: Move more into Sphinx, add external modules, mention MicroPython 1.25

Author: jonnor

README.md

@@ -17,9 +17,11 @@ Builds on [emlearn](https://emlearn.org), a C99 library for Machine Learning on
 ## Status
 **Minimally useful**
 
-- Tested *working* on `x64` (Unix port) and `xtensawin` (ESP32/ESP32-S3/etc).
-- Currently not supported: [armv6m/Cortex-M0/RP2040](https://github.com/emlearn/emlearn-micropython/issues/19)
-and [RISC-V/ESP32-C3/ESP32-C6](https://github.com/emlearn/emlearn-micropython/issues/35)
+- Initial set of Machine Learning and Digital Signal Processing modules available, including example projects.
+- Supports most MicroPython ports using runtime installable native modules
+- Primarily tested on `x64` (Unix port) and `xtensawin` (ESP32/ESP32-S3/etc).
+- Some devices only supported using external, notably armv6m/Cortex-M0/RP2040
+- Not yet supported: [RISC-V/ESP32-C3/ESP32-C6](https://github.com/emlearn/emlearn-micropython/issues/35)
 
 ## Features
 
@@ -35,7 +37,7 @@ and [RISC-V/ESP32-C3/ESP32-C6](https://github.com/emlearn/emlearn-micropython/is
 - Operates on standard `array.array` data structures
 - Models can be loaded at runtime from a file in disk/flash
 - Highly efficient. Inference times down to 100 microseconds, RAM usage <2 kB, FLASH usage <2 kB
-- Pre-built binaries available for most architectures.
+- Pre-built native modules available for most common architectures `xtensawin`.
 
 ## Examples
 
@@ -46,109 +48,36 @@ and [RISC-V/ESP32-C3/ESP32-C6](https://github.com/emlearn/emlearn-micropython/is
 
 ## Documentation
 
-Complete [documentation on ReadTheDocs](https://emlearn-micropython.readthedocs.io/en/latest/user_guide.html).
+Complete usage [documentation on ReadTheDocs](https://emlearn-micropython.readthedocs.io/en/latest/user_guide.html).
 
-## Prerequisites
 
-Minimally you will need
-
-- Python 3.10+ on host
-- MicroPython 1.24+ running onto your device
+## Citations
 
-#### Download repository
+If you use `emlearn-micropython` in an academic work, please reference it using:
 
-Download the repository with examples etc
-```
-git clone https://github.com/emlearn/emlearn-micropython
+```tex
+@misc{emlearn_micropython,
+  author       = {Jon Nordby},
+  title        = {{emlearn-micropython: Efficient Machine Learning engine for MicroPython}},
+  month        = aug,
+  year         = 2023,
+  doi          = {10.5281/zenodo.8212731},
+  url          = {https://doi.org/10.5281/zenodo.8212731}
+}
 ```
 
-## Usage
-
-Start with the instructions in [XOR example](./examples/xor_trees/).
-
-
-## Supported versions
-
-At any given point in time, emlearn-micropython only provides pre-built binaries for one MicroPython version.
-In general we strongly encourage people to use the latest version.
-There are no long-term-support or bugfix versions, at this point.
-If you build from source, the current version of emlearn-micropython might also work on a couple of MicroPython versions around the time, but this is not guaranteed.
-
-| MicroPython      | emlearn-micropython  |
-|------------------| ------------------   |
-| 1.24.x           | master               |
-| 1.24.x           | 0.7.0                |
-| 1.23.x           | 0.6.0                |
-
-#### Find architecture and .mpy version
-
-The correct .mpy files to use depend on the CPU architecture of your microcontroller,
-as well as the MicroPython version.
-
-| MicroPython version | .mpy version  |
-|---------------------| ------------- |
-| 1.23.x              | 6.3           |
-| 1.24.x              | 6.3           |
-
-
-Identify which CPU architecture your device uses.
-You need to specify `ARCH` to install the correct module version.
-
-| ARCH          | Description                       | Examples              |
-|---------------|-----------------------------------|---------------------- |
-| x64           | x86 64 bit                        | PC                    |
-| x86           | x86 32 bit                        |                       |
-| armv6m        | ARM Thumb (1)                     | Cortex-M0             |
-| armv7m        | ARM Thumb 2                       | Cortex-M3             |
-| armv7emsp     | ARM Thumb 2, single float         | Cortex-M4F, Cortex-M7 |
-| armv7emdp     | ARM Thumb 2, double floats        | Cortex-M7             |
-| xtensa        | non-windowed                      | ESP8266               |
-| xtensawin     | windowed with window size 8       | ESP32                 |
-
-Information is also available in the official documentation:
-[MicroPython: .mpy files](https://docs.micropython.org/en/latest/reference/mpyfiles.html#versioning-and-compatibility-of-mpy-files)
 
 
-## More learning resources
+## Developing
 
-emlearn-micropython and emlearn has been covered in the following presentations.
+For those that wish to hack on emlearn-micropython itself.
 
-- Microcontrollers + Machine Learning in 1-2-3 (PyData Global 2024).
-[Slides etc](https://github.com/jonnor/embeddedml/tree/master/presentations/PyDataGlobal2024)
-- Sensor data processing on microcontrollers with MicroPython and emlearn (PyConZA 2024).
-[Slides etc](https://github.com/jonnor/embeddedml/tree/master/presentations/PyConZA2024)
-- 6 years of open source TinyML with emlearn - a scikit-learn for microcontrollers (TinyML EMEA 2024)
-[YouTube video](https://www.youtube.com/watch?v=oG7PjPMA3Is) |
-[Slides etc](https://github.com/jonnor/embeddedml/tree/master/presentations/TinymlEMEA2024)
-- emlearn - Machine Learning for Tiny Embedded Systems (Embedded Online Conference 2024).
-[Youtube video](https://www.youtube.com/watch?v=qamVWmcBdmI) |
-[Slides etc](https://github.com/jonnor/embeddedml/tree/master/presentations/EmbeddedOnlineConference2024)
-- Machine Learning on microcontrollers using MicroPython and emlearn (PyCon DE & PyData Berlin 2024).
-[Slides etc](https://github.com/jonnor/embeddedml/tree/master/presentations/PyDataBerlin2024) |
-[YouTube video](https://www.youtube.com/watch?v=_MGm8sctqjg&t=1311s&pp=ygUSZW1sZWFybiBtaWNyb3B5dGhv).
+#### Download the code
 
-Here is an overview of resources for [TinyML in general](https://tinyml.seas.harvard.edu/courses/).
-
-## Benchmarks
-
-#### UCI handwriting digits
-
-UCI ML hand-written digits datasets dataset from
-[sklearn.datasets.load_digits](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_digits.html).
-8x8 image, 64 features. Values are 4-bit integers (16 levels). 10 classes.
-
-Running with a very simple RandomForest, 7 trees.
-Reaches approx 86% accuracy.
-Tested on Raspberry PI Pico, with RP2040 microcontroller (ARM Cortex M0 @ 133 MHz).
-
-![Inferences per second](./benchmarks/digits_bench.png)
-
-NOTE: over half of the time for emlearn case,
-is spent on converting the Python lists of integers into a float array.
-Removing that bottleneck would speed up things considerably.
-
-
-## Developing locally
+Clone the repository using git
+```
+git clone https://github.com/emlearn/emlearn-micropython
+```
 
 #### Prerequisites
 These come in addition to the prequisites described above.
@@ -159,13 +88,13 @@ See [MicroPython: Building native modules](https://docs.micropython.org/en/lates
 We assume that micropython is installed in the same place as this repository.
 If using another location, adjust `MPY_DIR` accordingly.
 
-You should be using MicroPython 1.24 (or newer).
+You should be using MicroPython 1.25 (or newer).
 
 #### Build
 
 Build the .mpy native module
 ```
-make dist ARCH=armv6m MPY_DIR=../micropython
+make dist ARCH=xtensawin MPY_DIR=../micropython
 ```
 
 Install it on device
@@ -180,18 +109,4 @@ To build and run tests on host
 make check
 ```
 
-## Citations
-
-If you use `emlearn-micropython` in an academic work, please reference it using:
-
-```tex
-@misc{emlearn_micropython,
-  author       = {Jon Nordby},
-  title        = {{emlearn-micropython: Efficient Machine Learning engine for MicroPython}},
-  month        = aug,
-  year         = 2023,
-  doi          = {10.5281/zenodo.8212731},
-  url          = {https://doi.org/10.5281/zenodo.8212731}
-}
-```
 

docs/external_modules.rst

@@ -0,0 +1,80 @@
+
+.. Places parent toc into the sidebar
+:parenttoc: True
+
+.. _external_modules:
+
+===============================
+External modules
+===============================
+
+An alternative, much simpler, installation method is as :ref:`native_modules` (recommended).
+
+This method is more complicated, as it requires building MicroPython from scratch.
+Therefore it is only recommended for 1) platforms which do not support native modules,
+and 2) advanced users.
+
+Supported versions
+===========================
+
+For general information about supported MicroPython versions, see :ref:`support`.
+In principle, external modules should work on any hardware/port of MicroPython.
+
+
+Prerequisites
+===========================
+
+External modules are included into the build process of MicroPython itself.
+That means that you must have a MicroPython build environment set up,
+including neccesary build toolchains et.c.
+This process is specific for each MicroPython port.
+Refer to the relevant port/X/README.md in the `micropython git repository <https://github.com/micropython/micropython/>`_ for the setup.
+
+Always make sure that you can succesfully build and run the vanilla firmware
+(no external modules), before you add in emlearn-micropython.
+
+You need a git checkout of emlearn-micropython.
+For example as a git submodule in your project.
+
+::
+    git clone https://github.com/emlearn/emlearn-micropython.git
+
+
+Include external modules in build
+===================================
+
+emlearn-micropython contains both C modules, as well as Python modules.
+Therefore it is neccesary to use both the options *USER_C_MODULES* and *FROZEN_MANIFEST*.
+
+For details on this process, see official MicroPython documentation on
+`building with external modules <https://docs.micropython.org/en/latest/develop/cmodules.html>`_.
+
+
+Example build
+----------------
+
+For platforms that use a cmake-based build system.
+
+::
+	make USER_C_MODULES=./emlearn-micropython/src/micropython.cmake \
+        FROZEN_MANIFEST=./emlearn-micropython/src/manifest.py \
+        CFLAGS_EXTRA='-Wno-unused-function -Wno-unused-function'
+
+
+For platforms that use a Makefile-based build.
+
+::
+
+	make USER_C_MODULES=./emlearn-micropython/src/ \
+        FROZEN_MANIFEST=./emlearn-micropython/src/manifest.py \
+        CFLAGS_EXTRA='-Wno-unused-function -Wno-unused-function'
+
+
+Combining multiple modules
+---------------------------
+
+If you want to add more C modules and/or frozen manifest,
+you will need to create your own `manifest.py` and `micropython.cmake` files.
+Use the ones in emlearn-micropython as a starting point.
+
+

docs/more.rst

@@ -24,6 +24,39 @@ This includes:
 Presentations
 ====================
 
+emlearn-micropython has been covered in the following presentations.
+
+
+Microcontrollers + Machine Learning in 1-2-3
+------------------------------------------------------------------
+
+Presented at PyData Global 2024, by Jon Nordby
+
+`Video recording (YouTube) <https://www.youtube.com/watch?v=nCmBJJHGQKo&pp=ygUUcHlkYXRhIGdsb2JhbCBub3JkYnk%3D>`_.
+
+`Slides and notes <https://github.com/jonnor/embeddedml/tree/master/presentations/PyDataGlobal2024>`_.
+
+
+MicroPython - Python for microcontrollers and embedded linux 
+-------------------------------------------------------------------------
+
+Presented at FOSDEM 2025, by Jon Nordby.
+Has a section on emlearn-micropython.
+
+`Video recording (YouTube) <https://www.youtube.com/watch?v=8Ao7DsTkpS4>`_.
+
+`Slides and notes <https://github.com/jonnor/embeddedml/tree/master/presentations/FOSDEM2025>`_.
+
+
+Sensor data processing on microcontrollers with MicroPython and emlearn
+-------------------------------------------------------------------------
+
+Presented at PyConZA 2024, by Jon Nordby.
+
+Video recording: coming...
+
+`Slides and notes <https://github.com/jonnor/embeddedml/tree/master/presentations/PyConZA2024>`_.
+
 
 Machine Learning on microcontrollers using MicroPython and emlearn
 ------------------------------------------------------------------
@@ -33,3 +66,10 @@ Presented at PyCon DE & PyData Berlin 2024, by Jon Nordby.
 `Video recording (YouTube) <https://www.youtube.com/watch?v=_MGm8sctqjg>`_.
 
 `Slides and notes <https://github.com/jonnor/embeddedml/tree/master/presentations/PyDataBerlin2024>`_.
+
+
+Other resources
+====================
+Here is an overview of resources for `TinyML in general <https://tinyml.seas.harvard.edu/courses/>`_.
+
+