SEVA: Leveraging sketches to evaluate alignment between human and machine visual abstraction

K Mukherjee, H Huey, X Lu, Y Vinker… - Advances in …, 2024 - proceedings.neurips.cc
Sketching is a powerful tool for creating abstract images that are sparse but meaningful.
Sketch understanding poses fundamental challenges for general-purpose vision algorithms …

Text-to-Image Diffusion Models are Great Sketch-Photo Matchmakers

S Koley, AK Bhunia, A Sain, PN Chowdhury… - arXiv preprint arXiv …, 2024 - arxiv.org
This paper, for the first time, explores text-to-image diffusion models for Zero-Shot Sketch-
based Image Retrieval (ZS-SBIR). We highlight a pivotal discovery: the capacity of text-to …

How to Handle Sketch-Abstraction in Sketch-Based Image Retrieval?

S Koley, AK Bhunia, A Sain, PN Chowdhury… - arXiv preprint arXiv …, 2024 - arxiv.org
In this paper, we propose a novel abstraction-aware sketch-based image retrieval
framework capable of handling sketch abstraction at varied levels. Prior works had mainly …

Zero-shot sketch-based image retrieval via adaptive relation-aware metric learning

Y Liu, Y Dang, X Gao, J Han, L Shao - Pattern Recognition, 2024 - Elsevier
Retrieving natural images with the query sketches under the zero-shot scenario is known as
zero-shot sketch-based image retrieval (ZS-SBIR). Most of the best-performing methods …

Causality-Invariant Interactive Mining for Cross-Modal Similarity Learning

J Yan, C Deng, H Huang, W Liu - IEEE Transactions on Pattern …, 2024 - ieeexplore.ieee.org
In the real world, how to effectively learn consistent similarity measurement across different
modalities is essential. Most of the existing similarity learning methods cannot deal well with …

A Systematic Literature Review of Deep Learning Approaches for Sketch-Based Image Retrieval: Datasets, Metrics, and Future Directions

F Yang, NA Ismail, YY Pang, VR Kebande… - IEEE …, 2024 - ieeexplore.ieee.org
Sketch-based image retrieval (SBIR) utilizes sketches to search for images containing
similar objects or scenes. Due to the proliferation of touch-screen devices, sketching has …

MagicLens: Self-Supervised Image Retrieval with Open-Ended Instructions

K Zhang, Y Luan, H Hu, K Lee, S Qiao, W Chen… - arXiv preprint arXiv …, 2024 - arxiv.org
Image retrieval, ie, finding desired images given a reference image, inherently
encompasses rich, multi-faceted search intents that are difficult to capture solely using …

Surgment: Segmentation-enabled Semantic Search and Creation of Visual Question and Feedback to Support Video-Based Surgery Learning

J Wang, H Tang, T Kantor, T Soltani, V Popov… - Proceedings of the CHI …, 2024 - dl.acm.org
Videos are prominent learning materials to prepare surgical trainees before they enter the
operating room (OR). In this work, we explore techniques to enrich the video-based surgery …

What sketch explainability really means for downstream tasks

H Bandyopadhyay, PN Chowdhury, AK Bhunia… - arXiv preprint arXiv …, 2024 - arxiv.org
In this paper, we explore the unique modality of sketch for explainability, emphasising the
profound impact of human strokes compared to conventional pixel-oriented studies. Beyond …

[HTML][HTML] Zero-Shot Sketch-Based Image Retrieval Using StyleGen and Stacked Siamese Neural Networks

VRMK Gopu, M Dunna - Journal of Imaging, 2024 - mdpi.com
Sketch-based image retrieval (SBIR) refers to a sub-class of content-based image retrieval
problems where the input queries are ambiguous sketches and the retrieval repository is a …