Visuo-haptic object perception for robots: an overview

N Navarro-Guerrero, S Toprak, J Josifovski… - Autonomous …, 2023 - Springer
The object perception capabilities of humans are impressive, and this becomes even more
evident when trying to develop solutions with a similar proficiency in autonomous robots …

Binding touch to everything: Learning unified multimodal tactile representations

F Yang, C Feng, Z Chen, H Park… - Proceedings of the …, 2024 - openaccess.thecvf.com
The ability to associate touch with other modalities has huge implications for humans and
computational systems. However multimodal learning with touch remains challenging due to …

Touching a nerf: Leveraging neural radiance fields for tactile sensory data generation

S Zhong, A Albini, OP Jones… - … on Robot Learning, 2023 - proceedings.mlr.press
Tactile perception is key for robotics applications such as manipulation. However, tactile
data collection is time-consuming, especially when compared to vision. This limits the use of …

Skill generalization of tubular object manipulation with tactile sensing and sim2real learning

Y Zhao, X Jing, K Qian, DF Gomes, S Luo - Robotics and Autonomous …, 2023 - Elsevier
Tubular objects such as test tubes are common in chemistry and life sciences research
laboratories, and robots that can handle them have the potential to accelerate experiments …

Unsupervised adversarial domain adaptation for sim-to-real transfer of tactile images

X Jing, K Qian, T Jianu, S Luo - IEEE Transactions on …, 2023 - ieeexplore.ieee.org
Transferring optical tactile skills learned from simulated environments to the real world
benefits many robotic tactile applications, which can reduce the cost of data collection …

Beyond flat gelsight sensors: Simulation of optical tactile sensors of complex morphologies for sim2real learning

DF Gomes, P Paoletti, S Luo - arXiv preprint arXiv:2305.12605, 2023 - arxiv.org
Recently, several morphologies, each with its advantages, have been proposed for the\textit
{GelSight} high-resolution tactile sensors. However, existing simulation methods are limited …

Marker or Markerless? Mode-Switchable Optical Tactile Sensing for Diverse Robot Tasks

N Ou, Z Chen, S Luo - IEEE Robotics and Automation Letters, 2024 - ieeexplore.ieee.org
Optical tactile sensors play a pivotal role in robot perception and manipulation tasks. The
membrane of these sensors can be painted with markers or remain markerless, enabling …

SingleS2R: Single sample driven Sim-to-Real transfer for Multi-Source Visual-Tactile Information Understanding using multi-scale vision transformers

J Tang, Z Gong, B Tao, Z Yin - Information Fusion, 2024 - Elsevier
Due to variations in light transmission and wear on the contact head, existing visual-tactile
dataset building methods typically require a large amount of real-world data, making the …

Bridging the simulation-to-real gap of depth images for deep reinforcement learning

Y Jang, J Baek, S Jeon, S Han - Expert Systems with Applications, 2024 - Elsevier
While deep reinforcement learning (DRL) models are effective at learning appropriate
actions from high-dimensional data, they require large amounts of costly and time …

Marker-embedded tactile image generation via generative adversarial networks

WD Kim, S Yang, W Kim, JJ Kim… - IEEE Robotics and …, 2023 - ieeexplore.ieee.org
Data-driven methods have been successfully applied to images from vision-based tactile
sensors to fulfill various manipulation tasks. Nevertheless, these methods remain inefficient …