Presentation
MAS-ISP: A Proxy-Free Online Hyperparameter Optimization Framework for ISP Hardware System
DescriptionThe rapid advancement of visual autonomous systems, especially in autonomous driving, underscores the critical role of Image Signal Processors (ISPs) as they convert RAW sensor data into RGB images suited for visual interpretation.
Traditional ISPs rely on tuning hyperparameters to adapt to varying imaging conditions;
however, the vast parameter space and intricate tuning process pose significant challenges for real-time autonomous applications.
Existing autonomous ISP hyperparameter optimization methods rely largely on offline or proxy-based online tuning, limiting their accuracy and responsiveness to real-time environmental changes.
In response, we propose an online ISP hyperparameter optimization framework based on Deep Reinforcement Learning (DRL), marking the first proxy-free, real-time optimization approach.
Our design exhibits a master-slave Multi-Agent System (MAS), enabling rapid and cooperative parameter optimization with improved inter-frame consistency.
Furthermore, we design the MAS-ISP automated visual system, incorporating innovative hardware designs such as Strip Conv Kernel and Stride-Aware Dual-Buffer Memory, which drastically reduce resource consumption in CNN hardware. MAS-ISP achieves 1080P@75FPS/240FPS on FPGA/ASIC platforms, supporting real-time and reliable visual systems.
Traditional ISPs rely on tuning hyperparameters to adapt to varying imaging conditions;
however, the vast parameter space and intricate tuning process pose significant challenges for real-time autonomous applications.
Existing autonomous ISP hyperparameter optimization methods rely largely on offline or proxy-based online tuning, limiting their accuracy and responsiveness to real-time environmental changes.
In response, we propose an online ISP hyperparameter optimization framework based on Deep Reinforcement Learning (DRL), marking the first proxy-free, real-time optimization approach.
Our design exhibits a master-slave Multi-Agent System (MAS), enabling rapid and cooperative parameter optimization with improved inter-frame consistency.
Furthermore, we design the MAS-ISP automated visual system, incorporating innovative hardware designs such as Strip Conv Kernel and Stride-Aware Dual-Buffer Memory, which drastically reduce resource consumption in CNN hardware. MAS-ISP achieves 1080P@75FPS/240FPS on FPGA/ASIC platforms, supporting real-time and reliable visual systems.
Event Type
Research Manuscript
TimeTuesday, June 242:45pm - 3:00pm PDT
Location3008, Level 3
SYS1: Autonomous Systems (Automotive, Robotics, Drones)
Similar Presentations