Model compression is one of the most popular approaches to improve the accessibility of
Large Language Models (LLMs) by reducing their memory footprint. However, the gaining of …

With the widespread adoption of deep learning (DL) applications in recent years, training DL
models has become increasingly prevalent. Nevertheless, training these models is typically …

Achieving efficient training has become one of the key factors affecting the popularization of
large model applications. The main technologies of efficient training of large models are …

With the development of emerging technologies such as cloud computing and large AI
models (such as LLM), many applications have placed higher demands on the intensive …

Recent years have witnessed a wide adoption of Remote Direct Memory Access (RDMA) to
accelerate distributed systems. As the scale of distributed applications keeps increasing …

Deep neural networks (DNNs) have achieved unparalleled performance in numerous fields,
including computer vision, natural language processing, and recommendation systems …

Training large language models (LLMs) demands increasingly larger datasets for optimal
performance [13]. In practice, these datasets may include hundreds of terabytes (TB) or even …