BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
X-LIC-LOCATION:America/Los_Angeles
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260402T024534Z
LOCATION:3001\, Level 3
DTSTART;TZID=America/Los_Angeles:20250623T134500
DTEND;TZID=America/Los_Angeles:20250623T140000
UID:dac_DAC 2025_sess108_RESEARCH757@linklings.com
SUMMARY:DRAFT: Decoupling Backpropagation from Pre-trained Backbone for Ef
 ficient Transformer Fine-Tuning on Edge
DESCRIPTION:Zhirui Huang and Shiwei Liu (Fudan University); Haozhe Zhu and
  Qi Liu (State Key Lab of Integrated Chips and Systems, Fudan University);
  and Chixiao Chen (Fudan University)\n\nTransformers have demonstrated out
 standing performance across diverse applications recently, necessitating f
 ine-tuning to optimize their performance for downstream tasks. However, fi
 ne-tuning remains challenging due to the substantial computational costs a
 nd storage overhead of backpropagation (BP). The existing fine-tuning tech
 niques require the BP through the massive pre-trained backbone weights for
  computing the input gradient, resulting in significant computing overhead
  and memory footprint for resource-constrained edge devices. To address th
 e challenge, this work proposes an algorithm-hardware co-design framework,
  DRAFT, for efficient Transformer fine-tuning by decoupling the BP from th
 e backbone weights, thereby efficiently reducing the BP overhead. The fram
 ework employs Feedback Decoupling Approximation (\textit{FDA}), an efficie
 nt fine-tuning algorithm that decouples BP into two low-complexity pathway
 s: trainable adapter pathway and sparse ternary Bypass Network (\textit{BP
 N}) pathway. The two pathways work collaboratively to approximate the conv
 entional BP process. Further, a DRAFT accelerator is proposed, featuring a
  reconfigurable design with lightweight sparse gather networks and dynamic
  workflows to fully harness the sparsity and data parallelism inherent to 
 the FDA. Experimental results demonstrate that DRAFT achieves a speedup of
  4.9× and an energy efficiency improvement of 4.2× on average compared to 
 baseline fine-tuning methods across multiple fine-tuning tasks with neglig
 ible accuracy loss.\n\nTopics: AI\n\nTracks: AI3: AI/ML Architecture Desig
 n\n\nSession Chairs: Abdelrahman Hosny (Apple Inc.) and Marina Neseem (Nvi
 dia)\n\n
END:VEVENT
END:VCALENDAR