Wireless sensor networking & embedded AI

Various applications demand more and more powerful machine inference in resource-scarce distributed devices. To allow intelligent applications at ultra-low energy and low latency, one needs 1.) custom AI processors, exploiting parallelism and data reuse under strong resource limitations; 2.) efficient ML models, optimized for the target hardware platform; 3.) data-efficient scheduling techniques and algorithm-to-hardware mapping tools. This talk will zoom into such a future of cross-layer optimized AI platforms for edge computing.

The Internet of Things (IoT) is rapidly materializing, but the growing volume of data generated by Far-Edge devices, often microcontroller-based, poses challenges for cloud-centric processing. TinyML addresses this challenge by enabling on-device ML inference, thereby reducing communication latency and cost. However, current solutions largely overlook deployment and management challenges, especially in heterogeneous, resource-constrained environments. This presentation introduces TinyKubeML, a Kubernetes-based framework that enables resource-aware deployment of TinyML models on Far-Edge clusters. It abstracts device heterogeneity and automates model partitioning, artifact generation, and deployment using a custom Kubernetes Operator. TinyKubeML supports distributed inference and includes recovery mechanisms to ensure service continuity. Our evaluation shows that TinyKubeML can deploy distributed models efficiently with minimal impact on accuracy, while supporting automatic recovery in the case of device failures, demonstrating its potential to bridge the gap between scalable orchestration and TinyML deployment in IoT scenarios.

Tiny Machine Learning (TinyML) allows the execution of small machine learning models on low-power devices like microcontrollers. TinyML-as-a-Service (TinyMLaaS) is an architecture to make the usage of TinyML models easier by having a platform that optimizes and compiles machine learning models according to the constraints of target devices, and then deploys the model code on microcontrollers. Within the Cloud-to-IoT continuum, both TinyML and multi-tenant microcontrollers focus on empowering microcontrollers and enabling on-device computing. Multi-tenant microcontrollers are designed to securely execute codes from mutually distrusting actors through the usage of lightweight software containerization solutions, like WebAssembly. We propose to integrate TinyMLaaS with multi-tenant microcontrollers by using WebAssembly-based containerization, and we implement a proof-of-concept of the TinyMLaaS architecture based on WebAssembly Micro Runtime (WAMR) and RIOT-ML. To improve the usage of containerized TinyML on microcontrollers, we propose CS4WAMR, a framework to enhance WAMR usage by enabling running simultaneously multiple instances of WAMR to allow better permission and memory consumption control.

Low-power multicore platforms are suitable for running data-intensive tasks in parallel, but they are highly inefficient for computing on intermittent power. We present PEARL (PowEr And eneRgy- aware MuLticore Intermittent Computing), a novel systems support that can make existing multicore microcontroller (MCU) platforms suitable for efficient intermittent computing. PEARL achieves this by leveraging only a three-threshold voltage tracking circuit and an external fast non-volatile memory, which multicore MCUs can smoothly interface. PEARL software runtime manages these components and performs energy- and power-aware adaptation of the multicore configuration to introduce minimal backup overheads and boost performance. Our evaluation shows that PEARL outperforms the state-of-the-art solutions by up to 30× and consumes up to 32× less energy.

• Demo 1 – NXP Semiconductors:
  embedded AI for audio applications
• Demo 2 – VersaSense:
  edge intelligence in practice: applications of condition monitoring in smart industries
• Demo 3 – Forcebit:
  high-throughput wireless sensing for linear & rotary drivetrains
• Demo 4 – Politecnico di Milano:
  on-the-fly extraction and compression of network traffic traces for efficient IoT forensics
• Demo 5 – Delft University of Technology:
  on-MCU traffic sign recognition with TinyML
• Demo 6 – Vanderbilt University:
  bringing networked sensing to high school

We will share our expertise with the implementation of ML algorithms on (heavily) resource-constrained embedded devices, and illustrate the different steps to realise efficient and robust implementations. We will focus on audio applications, but our approach and conclusions are generically applicable to other applications in the Artificial Intelligence of Things.