LightNobel: Improving Sequence Length Limitation in Protein Structure Prediction Model via Adaptive Activation Quantization

At the software level, LightNobel proposes Token-wise Adaptive Activation Quantization (AAQ), which leverages unique token-wise characteristics, such as distogrampatterns in PPM activations, to enable fine-grained quantization techniques without compromising accuracy. At the hardware level, LightNobel integrates the multi-precision reconfigurable matrix processing unit (RMPU) and versatile vector processing unit (VVPU) to enable the efficient execution of […]

ADOR: A Design Exploration Framework for LLM Serving with Enhanced Latency and Throughput

By leveraging predefined architecture templates specialized for heterogeneous dataflows, ADOR optimally balances throughput and latency. It efficiently explores design spaces to suggest architecturesthat meet the requirements of both vendors and users. ADOR demonstrates substantial performance improvements, achieving 2.51× higher QoS and 4.01× better area efficiency compared to the A100 at high batch sizes, making it […]