Evaluating the cybersecurity robustness of commercial llms against adversarial prompts: A promptbench analysis

T Goto, K Ono, A Morita - Authorea Preprints, 2024 - techrxiv.org
This study presents a comprehensive evaluation of the cybersecurity robustness of five
leading Large Language Models (LLMs)-ChatGPT-4, Google Gemini, Anthropic Claude …

A comparative analysis of large language models to evaluate robustness and reliability in adversarial conditions

T Goto, K Ono, A Morita - Authorea Preprints, 2024 - techrxiv.org
This study went on a comprehensive evaluation of four prominent Large Language Models
(LLMs)-Google Gemini, Mistral 8x7B, ChatGPT-4, and Microsoft Phi-1.5-to assess their …

Promptbench: A unified library for evaluation of large language models

K Zhu, Q Zhao, H Chen, J Wang, X Xie - arXiv preprint arXiv:2312.07910, 2023 - arxiv.org
The evaluation of large language models (LLMs) is crucial to assess their performance and
mitigate potential security risks. In this paper, we introduce PromptBench, a unified library to …

Assessing Adversarial Robustness of Large Language Models: An Empirical Study

Z Yang, Z Meng, X Zheng, R Wattenhofer - arXiv preprint arXiv …, 2024 - arxiv.org
Large Language Models (LLMs) have revolutionized natural language processing, but their
robustness against adversarial attacks remains a critical concern. We presents a novel white …

garak: A Framework for Security Probing Large Language Models

L Derczynski, E Galinkin, J Martin, S Majumdar… - arXiv preprint arXiv …, 2024 - arxiv.org
As Large Language Models (LLMs) are deployed and integrated into thousands of
applications, the need for scalable evaluation of how models respond to adversarial attacks …

Large Language Models in Cybersecurity: State-of-the-Art

F Nourmohammadzadeh Motlagh… - arXiv e …, 2024 - ui.adsabs.harvard.edu
Abstract The rise of Large Language Models (LLMs) has revolutionized our comprehension
of intelligence bringing us closer to Artificial Intelligence. Since their introduction …

Securing Large Language Models: Threats, Vulnerabilities and Responsible Practices

S Abdali, R Anarfi, CJ Barberan, J He - arXiv preprint arXiv:2403.12503, 2024 - arxiv.org
Large language models (LLMs) have significantly transformed the landscape of Natural
Language Processing (NLP). Their impact extends across a diverse spectrum of tasks …

ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teaming

S Tedeschi, F Friedrich, P Schramowski… - arXiv preprint arXiv …, 2024 - arxiv.org
When building Large Language Models (LLMs), it is paramount to bear safety in mind and
protect them with guardrails. Indeed, LLMs should never generate content promoting or …

[HTML][HTML] Guardian: A multi-tiered defense architecture for thwarting prompt injection attacks on llms

P Rai, S Sood, VK Madisetti, A Bahga - Journal of Software Engineering …, 2024 - scirp.org
This paper introduces a novel multi-tiered defense architecture to protect language models
from adversarial prompt attacks. We construct adversarial prompts using strategies like role …

Breaking down the defenses: A comparative survey of attacks on large language models

AG Chowdhury, MM Islam, V Kumar, FH Shezan… - arXiv preprint arXiv …, 2024 - arxiv.org
Large Language Models (LLMs) have become a cornerstone in the field of Natural
Language Processing (NLP), offering transformative capabilities in understanding and …