In recent years, advancements in smart home technologies have underscored the need for
the development of early fire and smoke detection systems to enhance safety and security …

Large Language Models (LLMs) like GPT and LLaMA are revolutionizing the AI industry with
their sophisticated capabilities. Training these models requires vast GPU clusters and …

Multimodal large language models (LLMs) have demonstrated significant potential in a wide
range of AI applications. Yet, training multimodal LLMs suffers from low efficiency and …

Offline batch inference, which leverages the flexibility of request batching to achieve higher
throughput and lower costs, is becoming more popular for latency-insensitive applications …

Given the popularity of generative AI, Large Language Models (LLMs) often consume
hundreds or thousands of GPUs for parallelizing and accelerating the training process …

Edge computing in federated learning based on centralized architecture often faces
communication constraints in large clusters. Although there have been some efforts like …

In recent years, Transformer-based Large Language Models (LLMs) have garnered
significant attention due to their exceptional performance across a variety of tasks. However …

To optimize large Transformer model training, efficient parallel computing and advanced
data management are essential. However, current methods often assume a stable and …

It is extremely memory-hungry to train Large Language Models (LLM). To solve this problem,
existing work exploits the combination of CPU and GPU for the training process, such as …

Training Transformer models on long sequences in a distributed setting poses significant
challenges in terms of efficiency and scalability. Current methods are either constrained by …