Reusing pretrained models by multi-linear operators for efficient training
Training large models from scratch usually costs a substantial amount of resources. Towards
this problem, recent studies such as bert2BERT and LiGO have reused small pretrained …
this problem, recent studies such as bert2BERT and LiGO have reused small pretrained …
Dq-lore: Dual queries with low rank approximation re-ranking for in-context learning
Recent advances in natural language processing, primarily propelled by Large Language
Models (LLMs), have showcased their remarkable capabilities grounded in in-context …
Models (LLMs), have showcased their remarkable capabilities grounded in in-context …
Tensor networks meet neural networks: A survey and future perspectives
Tensor networks (TNs) and neural networks (NNs) are two fundamental data modeling
approaches. TNs were introduced to solve the curse of dimensionality in large-scale tensors …
approaches. TNs were introduced to solve the curse of dimensionality in large-scale tensors …
Bayesian tensor network structure search and its application to tensor completion
Tensor network (TN) has demonstrated remarkable efficacy in the compact representation of
high-order data. In contrast to the TN methods with pre-determined structures, the recently …
high-order data. In contrast to the TN methods with pre-determined structures, the recently …
Expression syntax information bottleneck for math word problems
Math Word Problems (MWP) aims to automatically solve mathematical questions given in
texts. Previous studies tend to design complex models to capture additional information in …
texts. Previous studies tend to design complex models to capture additional information in …
A Highly compressed accelerator with temporal optical flow feature fusion and tensorized LSTM for video action recognition on terminal device
P Zhen, X Yan, W Wang, H Wei… - IEEE Transactions on …, 2023 - ieeexplore.ieee.org
Deep learning-based action recognition has become ubiquitous in the video analysis area;
however, large neural networks require enormous computations to achieve high …
however, large neural networks require enormous computations to achieve high …
An effective weight initialization method for deep learning: Application to satellite image classification
The growing interest in satellite imagery has triggered the need for efficient mechanisms to
extract valuable information from these vast data sources, providing deeper insights. Even …
extract valuable information from these vast data sources, providing deeper insights. Even …
Advocating for the Silent: Enhancing Federated Generalization for Nonparticipating Clients
Federated learning (FL) has surged in prominence due to its capability of collaborative
model training without direct data sharing. However, the vast disparity in local data …
model training without direct data sharing. However, the vast disparity in local data …
Low rank optimization for efficient deep learning: Making a balance between compact architecture and fast training
Deep neural networks (DNNs) have achieved great success in many data processing
applications. However, high computational complexity and storage cost make deep learning …
applications. However, high computational complexity and storage cost make deep learning …
Compute Better Spent: Replacing Dense Layers with Structured Matrices
Dense linear layers are the dominant computational bottleneck in foundation models.
Identifying more efficient alternatives to dense matrices has enormous potential for building …
Identifying more efficient alternatives to dense matrices has enormous potential for building …