Presentation
An Energy-Efficient High-Utilization Hardware Architecture for Attention Mechanism in Transformer using Balanced Systolic Array and Multi-Row Interleaved Operation Ordering
DescriptionDesigning energy-efficient, high-speed accelerators for attention mechanisms, which dominate energy and latency in Transformers, is increasingly significant. We observe two limitations in previous work: First, existing systolic arrays (SAs) fail to balance data reuse, register saving, and utilization. Second, layer-by-layer operation ordering incurs high SRAM access overhead for intermediate results. To address the first, we propose the "Balanced Systolic Array," improving energy efficiency (EE) by 40% with a 99.5% utilization rate. To address the second, we propose "Multi-Row Interleaved" operation ordering, reducing SRAM energy by 31.7%. Combining these techniques, the proposed accelerator improves EE by 39% over previous works.
Event Type
Research Manuscript
TimeMonday, June 232:30pm - 2:45pm PDT
Location3001, Level 3