Presentation
CognitiveArm: Enabling Real-Time EEG-Controlled Prosthetic Arm Using Embodied Machine Learning
DescriptionEfficient control of prosthetic limbs via non-invasive brain-computer interfaces (BCIs) requires advanced EEG processing capabilities—including pre-filtering, feature extraction, and action prediction—all performed in real-time on edge AI hardware. Achieving this level of real-time processing on resource-constrained edge devices presents significant challenges in balancing model complexity, computational efficiency, and latency. We present CognitiveArm, an EEG-driven, brain-controlled prosthetic system implemented on edge AI hardware, achieving real-time performance without compromising accuracy. The system integrates BrainFlow—an open-source library for EEG data acquisition and streaming—and optimized deep learning (DL) models for precise brain signal classification. By leveraging evolutionary search, we identify Pareto-optimal DL model configurations through hyper-parameter tuning, optimizer analysis, and window selection, analyzed individually and in ensemble configurations. We further apply model compression techniques like pruning and quantization to optimize these models for embedded deployment, balancing computational efficiency and accuracy. We collected EEG dataset and designed an annotation pipeline, enabling precise labeling of brain signals corresponding to specific intended actions, which forms the foundation for training our optimized deep learning (DL) models. Our CognitiveArm system also supports voice commands for seamless mode switching, enabling control of the prosthetic arm's 3 degrees of freedom (DoF). Running independently on embedded hardware, CognitiveArm ensures low latency and facilitates real-time interaction. We developed a full-scale prototype of the CognitiveArm, interfaced with the OpenBCI UltraCortex Mark IV EEG headset. Our evaluations demonstrate a significant improvement in accuracy, reaching up to 96%, for classifying three core actions (left, right, and stay idle). The integration of voice commands allows for multiplexed, variable movement, enabling multi-action control for various everyday tasks (e.g., handshake, cup picking). This enhances CognitiveArm's real-world performance for prosthetic control, demonstrating its potential as a practical solution for individuals requiring advanced prosthetic limb control.
Event Type
Research Manuscript
TimeTuesday, June 242:00pm - 2:15pm PDT
Location3008, Level 3
SYS1: Autonomous Systems (Automotive, Robotics, Drones)