Presentation
SnapPix: Efficient-Coding--Inspired In-Sensor Compression for Edge Vision
DescriptionEnergy-efficient image acquisition on the edge is
crucial for enabling remote sensing applications where the sensor
node has weak compute capabilities and must transmit data to
a remote server/cloud for processing. To reduce the edge energy
consumption, this paper proposes a sensor-algorithm co-designed
system called SnapPix, which compresses raw pixels in the
analog domain inside the sensor. We use coded exposure (CE)
as the in-sensor compression strategy as it offers the flexibility
to sample, i.e., selectively expose pixels, both spatially and
temporally. SnapPix has three contributions. First, we propose
a task-agnostic strategy to learn the sampling/exposure pattern
based on the classic theory of efficient coding. Second, we co-
design the downstream vision model with the exposure pattern to
address the pixel-level non-uniformity unique to CE-compressed
images. Finally, we propose lightweight augmentations to the
image sensor hardware to support our in-sensor CE compres-
sion. Evaluating on action recognition and video reconstruction,
SNAPPIX outperforms state-of-the-art video-based methods at
the same speed while reducing the energy by up to 15.4×.
crucial for enabling remote sensing applications where the sensor
node has weak compute capabilities and must transmit data to
a remote server/cloud for processing. To reduce the edge energy
consumption, this paper proposes a sensor-algorithm co-designed
system called SnapPix, which compresses raw pixels in the
analog domain inside the sensor. We use coded exposure (CE)
as the in-sensor compression strategy as it offers the flexibility
to sample, i.e., selectively expose pixels, both spatially and
temporally. SnapPix has three contributions. First, we propose
a task-agnostic strategy to learn the sampling/exposure pattern
based on the classic theory of efficient coding. Second, we co-
design the downstream vision model with the exposure pattern to
address the pixel-level non-uniformity unique to CE-compressed
images. Finally, we propose lightweight augmentations to the
image sensor hardware to support our in-sensor CE compres-
sion. Evaluating on action recognition and video reconstruction,
SNAPPIX outperforms state-of-the-art video-based methods at
the same speed while reducing the energy by up to 15.4×.
Event Type
Research Manuscript
TimeTuesday, June 2411:30am - 11:45am PDT
Location3000, Level 3
AI
AI4: AI/ML System and Platform Design