Understanding and improving visual prompting: A label-mapping perspective

A Chen, Y Yao, PY Chen… - Proceedings of the …, 2023 - openaccess.thecvf.com
We revisit and advance visual prompting (VP), an input prompting technique for vision tasks.
VP can reprogram a fixed, pre-trained source model to accomplish downstream tasks in the …

Fairness reprogramming

G Zhang, Y Zhang, Y Zhang, W Fan… - Advances in …, 2022 - proceedings.neurips.cc
Despite a surge of recent advances in promoting machine Learning (ML) fairness, the
existing mainstream approaches mostly require training or finetuning the entire weights of …

Text-visual prompting for efficient 2d temporal video grounding

Y Zhang, X Chen, J Jia, S Liu… - Proceedings of the IEEE …, 2023 - openaccess.thecvf.com
In this paper, we study the problem of temporal video grounding (TVG), which aims to predict
the starting/ending time points of moments described by a text sentence within a long …

Seasoning model soups for robustness to adversarial and natural distribution shifts

F Croce, SA Rebuffi, E Shelhamer… - Proceedings of the …, 2023 - openaccess.thecvf.com
Adversarial training is widely used to make classifiers robust to a specific threat or
adversary, such as l_p-norm bounded perturbations of a given p-norm. However, existing …

Visual prompting for adversarial robustness

A Chen, P Lorenz, Y Yao, PY Chen… - ICASSP 2023-2023 …, 2023 - ieeexplore.ieee.org
In this work, we leverage visual prompting (VP) to improve adversarial robustness of a fixed,
pre-trained model at test time. Compared to conventional adversarial defenses, VP allows …

Learning to learn from APIs: black-box data-free meta-learning

Z Hu, L Shen, Z Wang, B Wu… - … on Machine Learning, 2023 - proceedings.mlr.press
Data-free meta-learning (DFML) aims to enable efficient learning of new tasks by meta-
learning from a collection of pre-trained models without access to the training data. Existing …

Holistic adversarial robustness of deep learning models

PY Chen, S Liu - Proceedings of the AAAI Conference on Artificial …, 2023 - ojs.aaai.org
Adversarial robustness studies the worst-case performance of a machine learning model to
ensure safety and reliability. With the proliferation of deep-learning-based technology, the …

Deepzero: Scaling up zeroth-order optimization for deep model training

A Chen, Y Zhang, J Jia, J Diffenderfer, J Liu… - arXiv preprint arXiv …, 2023 - arxiv.org
Zeroth-order (ZO) optimization has become a popular technique for solving machine
learning (ML) problems when first-order (FO) information is difficult or impossible to obtain …

Revisiting zeroth-order optimization for memory-efficient llm fine-tuning: A benchmark

Y Zhang, P Li, J Hong, J Li, Y Zhang, W Zheng… - arXiv preprint arXiv …, 2024 - arxiv.org
In the evolving landscape of natural language processing (NLP), fine-tuning pre-trained
Large Language Models (LLMs) with first-order (FO) optimizers like SGD and Adam has …

ZooPFL: Exploring black-box foundation models for personalized federated learning

W Lu, H Yu, J Wang, D Teney, H Wang, Y Chen… - arXiv preprint arXiv …, 2023 - arxiv.org
When personalized federated learning (FL) meets large foundation models, new challenges
arise from various limitations in resources. In addition to typical limitations such as data …