Presentation

· Contributors · Organizations · Search Program · Flagged · Happening Now

Enabling On-Tiny-Device Model Personalization via Gradient Condensing and Alternant Partial Update

SessionProgram, Debug, Accelerate: Software Innovations

DescriptionOn-device training enables the model to adapt to user-specific data by fine-tuning a pre-trained model locally. As embedded devices become ubiquitous, on-device training is increasingly essential since users can benefit from the personalized model without transmitting data and model parameters to the server. Despite significant efforts toward efficient training, on-device training still faces two major challenges: (1) the heavy and offline computation workload required to identify important parameters for updating; (2) the prohibitive cost of multi-layer backpropagation, which strains the limited resources of embedded devices. In this paper, we propose an algorithm-system co-optimization framework that enables self-adaptive and on-device model personalization for resource-constrained embedded devices. To address the challenge of parameter selection, we propose Alternant Partial Update, a method that locally identifies essential parameters without requiring retraining or offline evolutionary search. To mitigate backpropagation costs, we introduce Gradient Condensing to condense the gradient map structure, significantly reducing the computational complexity and memory consumption of backpropagation while preserving model performance. Our framework is evaluated through extensive experiments using various CNN models (e.g., MobileNet, MCUNet) on embedded devices with minimal resources (e.g., OpenMV-H7 with less than 1MB SRAM and 2MB Flash). Experimental results show that our framework achieves up to 2X speedup, 80% memory saving, and 30% accuracy improvement on downstream tasks, outperforming SOTA approaches.

Authors