Presentation
Efficient Edge AI Learning with Equilibrium Propagation: A Practical Solution For Gradient Computation
DescriptionThe rapid growth of smart devices and sensors has led to an overwhelming
increase in data generation, pushing current network infrastructure to its
limits and threatening the scalability of cloud-based processing. Edge machine
learning, which processes data locally on devices, presents a viable solution
to reduce network load and latency. However, deploying deep learning at the
edge remains difficult due to the limited memory and computational capacity of
these devices which mostly precludes on-device/on-site training. Equilibrium
propagation (EP) has emerged as a promising alternative to backpropagation,
leveraging analog processing and device physics for energy-efficient learning.
Yet, its practical implementation is hindered by challenges such as voltage
variations and the need for energy-efficient circuits capable of gradient
computation at a sufficient level of accuracy. Existing solutions rely on
impractical idealized models. In this work, we introduce a novel method to
address the problem of the wide dynamic range of the voltage variation to avoid
the use of expensive low-noise amplifiers, and propose an innovative
transistor-level switched-capacitor circuit to compute gradients in accordance
with the EP rule. Additionally, our design supports batching, a key requirement
for stable training that is often overlooked. We validate our approach on the MNIST
dataset, demonstrating a practical, energy-efficient EP circuit that operates
within real hardware constraints.
increase in data generation, pushing current network infrastructure to its
limits and threatening the scalability of cloud-based processing. Edge machine
learning, which processes data locally on devices, presents a viable solution
to reduce network load and latency. However, deploying deep learning at the
edge remains difficult due to the limited memory and computational capacity of
these devices which mostly precludes on-device/on-site training. Equilibrium
propagation (EP) has emerged as a promising alternative to backpropagation,
leveraging analog processing and device physics for energy-efficient learning.
Yet, its practical implementation is hindered by challenges such as voltage
variations and the need for energy-efficient circuits capable of gradient
computation at a sufficient level of accuracy. Existing solutions rely on
impractical idealized models. In this work, we introduce a novel method to
address the problem of the wide dynamic range of the voltage variation to avoid
the use of expensive low-noise amplifiers, and propose an innovative
transistor-level switched-capacitor circuit to compute gradients in accordance
with the EP rule. Additionally, our design supports batching, a key requirement
for stable training that is often overlooked. We validate our approach on the MNIST
dataset, demonstrating a practical, energy-efficient EP circuit that operates
within real hardware constraints.
Event Type
Networking
Work-in-Progress Poster
TimeMonday, June 236:00pm - 7:00pm PDT
LocationLevel 2 Lobby
Similar Presentations