GelStereo Visuotactile Perception for Dexterous Robotic Manipulation
Date:
Visuotactile sensing plays a crucial role in advancing robotic dexterity by providing high-resolution contact information. However, the development of high-fidelity visuotactile sensors and simulators remains a significant challenge. In this talk, I will introduce our work on GelStereo, a series of biomimetic visuotactile sensors capable of high-precision 3D contact deformation sensing on multi-curvature surfaces. Compared to traditional visuotactile sensors such as GelSight, GelStereo significantly reduces optical and structural complexity while enhancing integration flexibility. I will also present TacFlex, an efficient and flexible visuotactile simulation framework based on Finite Element Methods (FEM) and ray-tracing rectification. TacFlex enables realistic tactile imprint generation for various sensor configurations, including arbitrary coating patterns and 3D tactile point clouds. Furthermore, we explore Sim2Real transfer in robotic manipulation tasks such as in-hand pose estimation and peg-in-hole assembly, demonstrating how our simulated tactile imprints facilitate learning-based robotic skills that transfer effectively to the real world.