Tiny Machine Learning for Resource-Constrained IoT Devices: Efficient Architectures, Quantization, and Hardware-Aware Inference

Abstract

The proliferation of low-power sensors and Internet of Things devices has created increasing demand for machine-learning inference directly on microcontrollers and other resource-constrained hardware. Conventional deep neural networks frequently exceed the memory, computation, latency, and energy budgets available on such platforms. Tiny machine learning, commonly termed TinyML, seeks to enable useful machine intelligence within kilobyte- to megabyte-scale memory environments while minimizing dependence on continuous cloud connectivity. This review examines the development of efficient embedded machine learning through 2021. Lightweight neural architectures including MobileNet, MobileNetV2, MobileNetV3, SqueezeNet, ShuffleNet, and EfficientNet are discussed together with model compression, pruning, low-precision inference, binary networks, quantization, neural architecture search, and hardware-aware optimization. Particular attention is given to MCUNet, which jointly designed neural architectures and an inference engine for microcontroller constraints, as well as Once-for-All and ProxylessNAS approaches for hardware-specialized deployment. Integer-only inference and mixed-precision quantization are evaluated as mechanisms for reducing memory and computational requirements. The review considers the trade-offs among model accuracy, SRAM consumption, flash storage, latency, energy efficiency, and device portability. Challenges include fragmented hardware ecosystems, limited benchmark standardization, memory bottlenecks, operator support, and differences between theoretical operation counts and real-device performance. The evidence available by 2021 suggested that hardware-software co-design would be central to making always-on artificial intelligence practical on very small IoT devices.