Presentation
SOFTONIC: A Photonic Design Approach to SoftMax Activation for High-Speed Analog AI Acceleration
DescriptionRecent advancements in Silicon Photonics (SiPh)-based AI accelera-
tors present promising solutions to address the energy bottlenecks
of executing large models, like Transformers. However, existing
SiPh solutions focus primarily on accelerating matrix multiplica-
tion (MM) in the photonic domain, while the Softmax Activation
(SMA) function—an operation that accounts for 30-40% of total
computation in Transformers—remains on conventional digital
platforms. This reliance leads to significant energy and latency
overheads due to frequent data conversions between photonic MM
and digital SMA. Several electro-optic and all-optical methods have
been developed to implement nonlinear activation functions (e.g.
ReLU, Sigmoid, Tanh, and Softplus) using Optical Amplifiers along-
side Mach-Zehnder Modulators (MZMs) or Microring Resonators
(MRRs). However, similar approaches are unsuitable for SMA due
to the excessive area and energy overhead introduced by optical
amplifiers. Additionally, devices like MRRs and MZMs alone are in-
sufficient for SMA's nonlinear operations (exponential and division
functions), as they are bound by Maxwell's equations. Consequently,
a photonics-compatible architecture for efficient implementation of
SMA remains unachieved due to its intricate nature. To address this,
we propose SOFTONIC—a first-of-its-kind photonic SMA architec-
ture designed to enable all-photonic acceleration of Transformer for
ultra high energy efficiency and speedup. Our approach leverages
range reduction techniques to adjust input domains and applies
Chebyshev polynomial approximations for efficient computation.
Simulations of SOFTONIC using industry-standard CAD tools and
AI workloads demonstrate a 109x improvement in latency and an
80% reduction in power consumption compared to leading digital
and analog Softmax hardware solutions, with minimal area over-
head.
tors present promising solutions to address the energy bottlenecks
of executing large models, like Transformers. However, existing
SiPh solutions focus primarily on accelerating matrix multiplica-
tion (MM) in the photonic domain, while the Softmax Activation
(SMA) function—an operation that accounts for 30-40% of total
computation in Transformers—remains on conventional digital
platforms. This reliance leads to significant energy and latency
overheads due to frequent data conversions between photonic MM
and digital SMA. Several electro-optic and all-optical methods have
been developed to implement nonlinear activation functions (e.g.
ReLU, Sigmoid, Tanh, and Softplus) using Optical Amplifiers along-
side Mach-Zehnder Modulators (MZMs) or Microring Resonators
(MRRs). However, similar approaches are unsuitable for SMA due
to the excessive area and energy overhead introduced by optical
amplifiers. Additionally, devices like MRRs and MZMs alone are in-
sufficient for SMA's nonlinear operations (exponential and division
functions), as they are bound by Maxwell's equations. Consequently,
a photonics-compatible architecture for efficient implementation of
SMA remains unachieved due to its intricate nature. To address this,
we propose SOFTONIC—a first-of-its-kind photonic SMA architec-
ture designed to enable all-photonic acceleration of Transformer for
ultra high energy efficiency and speedup. Our approach leverages
range reduction techniques to adjust input domains and applies
Chebyshev polynomial approximations for efficient computation.
Simulations of SOFTONIC using industry-standard CAD tools and
AI workloads demonstrate a 109x improvement in latency and an
80% reduction in power consumption compared to leading digital
and analog Softmax hardware solutions, with minimal area over-
head.
Event Type
Networking
Work-in-Progress Poster
TimeSunday, June 226:00pm - 7:00pm PDT
LocationLevel 3 Lobby
Similar Presentations