Presentation
eMamba: Efficient Acceleration of Mamba Models for Edge Computing
DescriptionState Space Model (SSM)-based machine learning architectures have recently gained attention for processing sequential data. A recent sequence-to-sequence SSM, Mamba, offers competitive accuracy over state-of-the-art transformers with higher processing efficiency. Competitive performance with lower complexity makes Mamba a compelling choice for edge ML applications. However, no hardware accelerator design frameworks have been specifically optimized for Mamba in edge scenarios to date. To fill this gap, we propose eMamba, an end-to-end framework for designing and deploying Mamba hardware accelerators for edge applications. eMamba enhances efficiency by replacing complex normalization layers with hardware-friendly alternatives while also approximating SiLU activation and exponent calculations. It also employs an approximation-aware Neural Architecture Search (NAS) to identify the best hyperparameters for edge deployment. The entire design is quantized and evaluated on AMD-ZCU102 using a mmWave radar-based human pose estimation application. eMamba achieves 9.95× higher throughput and 5.62× lower latency using 63× fewer parameters while maintaining competitive accuracy with state-of-the-art solutions.
Event Type
Networking
Work-in-Progress Poster
TimeMonday, June 236:00pm - 7:00pm PDT
LocationLevel 2 Lobby