BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
X-LIC-LOCATION:America/Los_Angeles
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260402T024534Z
LOCATION:3001\, Level 3
DTSTART;TZID=America/Los_Angeles:20250623T140000
DTEND;TZID=America/Los_Angeles:20250623T141500
UID:dac_DAC 2025_sess108_RESEARCH319@linklings.com
SUMMARY:Move Less, Retrieve Fast: A Retrieval-in-Memory Architecture for L
 anguage Models
DESCRIPTION:Jiaxian Chen, Yuxuan Qi, Jianan Yuan, Kaoyi Sun, Tianyu Wang, 
 Chenlin Ma, and Yi Wang (Shenzhen University)\n\nRetrieval-augmented langu
 age models (RALMs) have attracted widespread attention for addressing the 
 limitations of traditional large language models. However, challenges invo
 lved in retrieval, including substantial data movement and irregular acces
 s patterns, seriously impact the efficiency and deployment of RALMs. The e
 merging 3D-stacked processing-in-memory (PIM) architecture, characterized 
 by its high memory bandwidth and near-data computing capabilities, present
 s a promising solution for efficient retrieval. To support large-scale ret
 rieval in RALMs, the PIM architecture should be carefully designed with jo
 int software and hardware optimization.\n\nThis paper presents Rimast, a r
 etrieval-in-memory architecture for fast retrieval in RALMs. The objective
  is to minimize data movement and improve overall performance through hard
 ware-software co-design. At the hardware level, a hierarchical PIM archite
 cture with a retrieval-in-memory dataflow is designed to reduce unnecessar
 y data transfer. At the software level, skew-free data mapping and adaptiv
 e offloading strategies are proposed to address the irregular access patte
 rns associated with retrieval in RALMs. We demonstrate the effectiveness o
 f the proposed Rimast using extensive experiments. The experimental result
 s demonstrate that Rimast effectively reduces data movement, achieving ave
 rage speedups of 273×, 55×, and 2.41× over CPUs, GPUs, and prior art accel
 erators, respectively.\n\nTopics: AI\n\nTracks: AI3: AI/ML Architecture De
 sign\n\nSession Chairs: Abdelrahman Hosny (Apple Inc.) and Marina Neseem (
 Nvidia)\n\n
END:VEVENT
END:VCALENDAR