LiME-TM

LiME-TM: A Lightning-fast and Memory-Efficient ML Framework
for On-Device Training on MCUs

Han Wu, Shengyu Duan and Tousif

Why is On-Device Training Important?

Data Distribution Shift

Evolving Environment
Sensor Degradation
User Personalization

The main reason is data distribution shift. If we train a model on a PC and then deploy the model on MCUs, the real-world data distribution may not be the same as the training data for several reasons: the environment is evolving, sensors will degrade over time, and people are different. For example, in the EMG dataset, the project Shengyu has been working on. The sensor measures the electrical activity of muscles and predicts the hand gesture. If we train the model on the server using data from a group of users, the model accuracy is 85%, but if we continue to train the model on the device using personal data, the accuracy can be improved to 96%. If we personalize the model for each user, we can achieve much better performance. The problem here is personal data is sensitive, and it cannot be uploaded to the server for training. We need to train on the device.

Why is On-Device Training Challenging?

Resource Constraints

Memory: GBs to KBs
Compute: GHz to MHz
Energy: W to mW

The main challenge is resource constraints. For example, my laptop has 8GB of memory, but a MCU has only KB of memory. My laptop has a 3GHz CPU, but a MCUs run at MHz. My laptop or cloud servers also has a stable power supply, but a MCU is usually powered by a small battery, so its performance is way worse than a laptop or cloud servers. As a result, TensorFlow Lite only support training on IOS, Android, but not MCUs. You know, IOS and Android uses Unix and Linux Operating System, there is no way to run Linux on a MCU, such as ESP32. We also did a review of on-device training frameworks, and as you can see here, our framework requires much less memory. You may wonder, this one seems to be the best, but it is a very small neural network with only 1 hidden layer and 10 neurons. The accuracy won't be very high. You know, theoretically we can have a neural network with only 1 neuron, it is memory efficient but not very helpful. For a useful 2D conv NN, it requires much more memory. Our framework does not hurt the accuracy. For inference, we have achieved state-of-the-art.

Can Vanilla TM do On-Device Training?

Vanilla TM

MNIST Dataset (60k images)

100 clauses
10 classes
28 x 28 pixels
8 bits

100 clauses x 10 classes x 28 x 28 pixels x 8 bits
= 12,544,000 Bytes (~ 12MB)

LiME-TM: Model Pruning

LiME-TM: Model Compiler

LiME-TM: Model Compression

LiME-TM: Bare-metal Support

LiME-TM: Inference Only

Thanks

https://lime-tm.wuhanstudio.uk

On-Device Training: 20 KB SRAM
On-Device Inference: 2 KB SRAM

Source Code