Presentation
XShift: FPGA-efficient Binarized LLM with Joint Quantization and Sparsification
DescriptionBinarization is a promising approach to significantly reduce computational complexity by replacing multiplications with hardware-efficient XNOR operations. However, the binarization of LLM activations often leads to severe accuracy degradation, while weight-only binarization fails to eliminate multipliers due to the Self-Attention mechanism. Furthermore, LLMs exhibit distinctive channel-level data distribution characteristics and differing computational and memory requirements between the Pre-fill and Decoding stages, necessitating a specialized inference framework.
In response, we introduce \textit{XShift}, an algorithm-hardware co-design framework optimized for efficient binarized LLM inference on FPGAs. \textit{XShift} incorporates three key contributions: (1) a hardware-friendly XNOR-Shift Encoding (XSE) format that transforms traditional multiplications into XNOR and shift operations, ensuring scalability and precision; (2) Hardware Adaptive Outlier and Sparsity (HAOS) techniques, which exploit channel-level data distribution and systolic array architectures for optimized quantization and sparsification; and (3) a dedicated hardware accelerator featuring an XNOR-Shift Systolic Array (XSSA) and an enhanced Base-2 SoftMax Converter (BSMC), designed to address the specific computational demands of binarized LLMs.
Experimental evaluations on the Alveo U280 and U50 FPGA demonstrate that \textit{\method} achieves a 10-15× reduction in DSP resource usage while surpassing existing accelerators and GPUs in inference performance. Specifically, \textit{XShift} delivers an average speedup of 4.17-4.76× and a 14.29-6.95× improvement in energy efficiency, alongside lower perplexity compared to other low-precision LLM techniques. These results underscore the potential of \textit{\method} for edge deployment of LLMs.
In response, we introduce \textit{XShift}, an algorithm-hardware co-design framework optimized for efficient binarized LLM inference on FPGAs. \textit{XShift} incorporates three key contributions: (1) a hardware-friendly XNOR-Shift Encoding (XSE) format that transforms traditional multiplications into XNOR and shift operations, ensuring scalability and precision; (2) Hardware Adaptive Outlier and Sparsity (HAOS) techniques, which exploit channel-level data distribution and systolic array architectures for optimized quantization and sparsification; and (3) a dedicated hardware accelerator featuring an XNOR-Shift Systolic Array (XSSA) and an enhanced Base-2 SoftMax Converter (BSMC), designed to address the specific computational demands of binarized LLMs.
Experimental evaluations on the Alveo U280 and U50 FPGA demonstrate that \textit{\method} achieves a 10-15× reduction in DSP resource usage while surpassing existing accelerators and GPUs in inference performance. Specifically, \textit{XShift} delivers an average speedup of 4.17-4.76× and a 14.29-6.95× improvement in energy efficiency, alongside lower perplexity compared to other low-precision LLM techniques. These results underscore the potential of \textit{\method} for edge deployment of LLMs.
Event Type
Research Manuscript
TimeTuesday, June 244:45pm - 5:00pm PDT
Location3000, Level 3
AI
AI3: AI/ML Architecture Design