A systematic survey of prompt engineering on vision-language foundation models

J Gu, Z Han, S Chen, A Beirami, B He, G Zhang… - arXiv preprint arXiv …, 2023 - arxiv.org
Prompt engineering is a technique that involves augmenting a large pre-trained model with
task-specific hints, known as prompts, to adapt the model to new tasks. Prompts can be …

On the design fundamentals of diffusion models: A survey

Z Chang, GA Koulieris, HPH Shum - arXiv preprint arXiv:2306.04542, 2023 - arxiv.org
Diffusion models are generative models, which gradually add and remove noise to learn the
underlying distribution of training data for data generation. The components of diffusion …

A survey on transferability of adversarial examples across deep neural networks

J Gu, X Jia, P de Jorge, W Yu, X Liu, A Ma… - arXiv preprint arXiv …, 2023 - arxiv.org
The emergence of Deep Neural Networks (DNNs) has revolutionized various domains,
enabling the resolution of complex tasks spanning image recognition, natural language …

On the robustness of latent diffusion models

J Zhang, Z Xu, S Cui, C Meng, W Wu… - arXiv preprint arXiv …, 2023 - arxiv.org
Latent diffusion models achieve state-of-the-art performance on a variety of generative tasks,
such as image synthesis and image editing. However, the robustness of latent diffusion …

Toward effective protection against diffusion-based mimicry through score distillation

H Xue, C Liang, X Wu, Y Chen - The Twelfth International …, 2023 - openreview.net
While generative diffusion models excel in producing high-quality images, they can also be
misused to mimic authorized images, posing a significant threat to AI systems. Efforts have …

Assessing robustness via score-based adversarial image generation

M Kollovieh, L Gosch, Y Scholten, M Lienen… - arXiv preprint arXiv …, 2023 - arxiv.org
Most adversarial attacks and defenses focus on perturbations within small $\ell_p $-norm
constraints. However, $\ell_p $ threat models cannot capture all relevant semantic …

Intriguing Properties of Diffusion Models: An Empirical Study of the Natural Attack Capability in Text-to-Image Generative Models

T Sato, J Yue, N Chen, N Wang… - Proceedings of the …, 2024 - openaccess.thecvf.com
Denoising probabilistic diffusion models have shown breakthrough performance to generate
more photo-realistic images or human-level illustrations than the prior models such as …

Instruct2Attack: Language-Guided Semantic Adversarial Attacks

J Liu, C Wei, Y Guo, H Yu, A Yuille, S Feizi… - arXiv preprint arXiv …, 2023 - arxiv.org
We propose Instruct2Attack (I2A), a language-guided semantic attack that generates
semantically meaningful perturbations according to free-form language instructions. We …

AdvDenoise: Fast Generation Framework of Universal and Robust Adversarial Patches Using Denoise

J Li, Z Wang, J Li - … of the IEEE/CVF Conference on …, 2024 - openaccess.thecvf.com
Adversarial patch attacks which can mislead deep learning models and the human eye in
both the digital and physical domains have led to a trust crisis. Traditional approaches to …

Zero-shot image harmonization with generative model prior

J Chen, Z Zou, Y Zhang, K Chen, Z Shi - arXiv preprint arXiv:2307.08182, 2023 - arxiv.org
Recent image harmonization methods have demonstrated promising results. However, due
to their heavy reliance on a large number of composite images, these works are expensive …