BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
X-LIC-LOCATION:America/Los_Angeles
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260402T024534Z
LOCATION:3001\, Level 3
DTSTART;TZID=America/Los_Angeles:20250625T153000
DTEND;TZID=America/Los_Angeles:20250625T154500
UID:dac_DAC 2025_sess132_RESEARCH680@linklings.com
SUMMARY:DIAS: Distance-based Attention Sparsity for Ultra-Long-Sequence Tr
 ansformer with Tree-like Processing-in-Memory Architecture
DESCRIPTION:Zekai Chen, Yiming Chen, Teng Wan, Tianyi Yu, Yu Wang, Huazhon
 g Yang, and Xueqing Li (Tsinghua University)\n\nLong-context inference has
  become a central focus in recent self-regressive Transformer research. Ho
 wever, challenges still remain in performing decode stage due to the compu
 tational complexity of attention mechanisms and the substantial overhead a
 ssociated with KV cache storage. Although attention sparsity has been prop
 osed as a potential solution, conventional sparsity methods that rely on h
 euristic algorithms often suffer from accuracy degradation when applied to
  ultra-long sequences.\n\nTo address the dilemma between accuracy and perf
 ormance, this work proposes DIAS, a distance-based irregular attention spa
 rsity approach with processing-in-memory (PIM) architecture. DIAS employs 
 top-K approximate attention scores through graph-based search to enhance i
 nference efficiency while maintaining accuracy. Furthermore, a scalable ga
 ther-and-scatter-based PIM architecture is introduced to manage the storag
 e demands of large-scale KV cache and to facilitate efficient sparsity att
 ention computations. Various configurations of DIAS evaluated on Longbench
  with Mistral and Llama3 models show xx times speedup and xx times energy 
 efficiency improvement, with accuracy drop of less than 1\%.\n\nTopics: De
 sign\n\nTracks: DES2B: In-memory and Near-memory Computing Architectures, 
 Applications and Systems\n\nSession Chairs: Steve Dai (NVIDIA) and Haitong
  Li (Purdue University)\n\n
END:VEVENT
END:VCALENDAR