Applications of transformer-based language models in bioinformatics: a survey

S Zhang, R Fan, Y Liu, S Chen, Q Liu… - Bioinformatics …, 2023 - academic.oup.com
The transformer-based language models, including vanilla transformer, BERT and GPT-3,
have achieved revolutionary breakthroughs in the field of natural language processing …

[HTML][HTML] Advancing drug discovery with deep attention neural networks

A Lavecchia - Drug Discovery Today, 2024 - Elsevier
In the dynamic field of drug discovery, deep attention neural networks are revolutionizing our
approach to complex data. This review explores the attention mechanism and its extended …

Attention is all you need: utilizing attention in AI-enabled drug discovery

Y Zhang, C Liu, M Liu, T Liu, H Lin… - Briefings in …, 2024 - academic.oup.com
Recently, attention mechanism and derived models have gained significant traction in drug
development due to their outstanding performance and interpretability in handling complex …

HiGNN: A hierarchical informative graph neural network for molecular property prediction equipped with feature-wise attention

W Zhu, Y Zhang, D Zhao, J Xu… - Journal of Chemical …, 2022 - ACS Publications
Elucidating and accurately predicting the druggability and bioactivities of molecules plays a
pivotal role in drug design and discovery and remains an open challenge. Recently, graph …

A fingerprints based molecular property prediction method using the BERT model

N Wen, G Liu, J Zhang, R Zhang, Y Fu… - Journal of Cheminformatics, 2022 - Springer
Molecular property prediction (MPP) is vital in drug discovery and drug reposition. Deep
learning-based MPP models capture molecular property-related features from various …

From black boxes to actionable insights: a perspective on explainable artificial intelligence for scientific discovery

Z Wu, J Chen, Y Li, Y Deng, H Zhao… - Journal of Chemical …, 2023 - ACS Publications
The application of Explainable Artificial Intelligence (XAI) in the field of chemistry has
garnered growing interest for its potential to justify the prediction of black-box machine …

Transformer technology in molecular science

J Jiang, L Ke, L Chen, B Dou, Y Zhu… - Wiley …, 2024 - Wiley Online Library
A transformer is the foundational architecture behind large language models designed to
handle sequential data by using mechanisms of self‐attention to weigh the importance of …

FG-BERT: a generalized and self-supervised functional group-based molecular representation learning framework for properties prediction

B Li, M Lin, T Chen, L Wang - Briefings in Bioinformatics, 2023 - academic.oup.com
Artificial intelligence-based molecular property prediction plays a key role in molecular
design such as bioactive molecules and functional materials. In this study, we propose a self …

Transformers for molecular property prediction: Lessons learned from the past five years

A Sultan, J Sieg, M Mathea… - Journal of Chemical …, 2024 - ACS Publications
Molecular Property Prediction (MPP) is vital for drug discovery, crop protection, and
environmental science. Over the last decades, diverse computational techniques have been …

Machine Learning Empowering Drug Discovery: Applications, Opportunities and Challenges

X Qi, Y Zhao, Z Qi, S Hou, J Chen - Molecules, 2024 - mdpi.com
Drug discovery plays a critical role in advancing human health by developing new
medications and treatments to combat diseases. How to accelerate the pace and reduce the …