publications

2026

1. arxiv

   

   Blind Dexterous Grasping via Real2Sim2Real Tactile Policy Learning

   Shengcheng Luo, Xiyan Huang, Zhe Xu, Wanlin Li, Ziyuan Jiao, and Chenxi Xiao

   2026

   Abs Bib PROJECT

   Blind grasping with a dexterous hand is a crucial manipulation capability. Nevertheless, learning such tactile-only policies for real robots remains challenging due to the tactile sim-to-real gap and the limited expressiveness of sparse tactile signals. To bridge this gap, we propose a framework for tactile-only blind grasping that is deployable on a physical multi-fingered robotic hand. Our approach combines three key components. First, we introduce a Real2Sim tactile calibration pipeline that constructs a contact-calibrated digital-twin simulator capable of reproducing real tactile signals. Second, we improve the expressiveness of sparse tactile observations using a layout-aware tactile encoder, which incorporates sensor-geometry priors through self-supervised pretraining. Third, to improve generalization to unseen objects, we train object-specific reinforcement-learning experts in the calibrated simulator and aggregate their successful grasp trajectories into a tactile-conditioned Diffusion Policy. We evaluate our method on a physical LEAP Hand equipped with distributed tactile sensing across 10 seen and 10 unseen objects. The deployed policy achieves a 27% real-world grasp success rate across all 20 objects, without real-world grasping demonstrations or visual input. Simulation ablations show that layout-aware tactile pretraining improves grasping performance, while sensing-level evaluations confirm that Real2Sim calibration increases the consistency of tactile contact events between simulation and hardware. Together, these results suggest that contact-event calibration, geometry-aware tactile representation learning, and diffusion-based policy aggregation provide an effective path toward tactile-only blind grasping on real dexterous robotic hands.

   @misc{luo2026blinddexterousgraspingreal2sim2real,
     title = {Blind Dexterous Grasping via Real2Sim2Real Tactile Policy Learning},
     author = {Luo, Shengcheng and Huang, Xiyan and Xu, Zhe and Li, Wanlin and Jiao, Ziyuan and Xiao, Chenxi},
     year = {2026},
     eprint = {2606.11767},
     archiveprefix = {arXiv},
     primaryclass = {cs.RO},
     url = {https://arxiv.org/abs/2606.11767},
   }

2. arxiv

   

   RGB-S: Image-Aligned Tactile Saliency for Robust Dexterous Manipulation

   Shengcheng Luo^*, Kefei Wu^*, Xiaoying Zhou, Wanlin Li, Ziyuan Jiao, and Chenxi Xiao

   2026

   Abs Bib PROJECT

   Effective visuo-tactile integration is critical for robotic dexterous manipulation, especially when visual observations are unreliable or occluded. However, robustly aligning sparse, heterogeneous tactile measurements with dense visual representations remains a fundamental challenge. Most existing approaches require policies to learn cross-modal correspondences implicitly from limited demonstrations, without leveraging geometric priors. As a result, they are often data-inefficient and generalize poorly when visual observations are degraded. To address this limitation, we propose a framework that explicitly grounds physical contacts in the image domain. Using robot forward kinematics and camera calibration, we project tactile sensor locations directly onto the RGB image plane. We then render force-modulated Gaussian saliency maps to model spatial uncertainty arising from kinematic and calibration errors. By integrating these 2D spatial anchors through a zero-initialized conditioning architecture, our method injects physical contact priors into standard visual backbones while preserving pre-trained visual representations. We evaluate our method on six dexterous manipulation tasks in both simulation and the real world under severe visual occlusions. Real-world experiments show that explicit RGB-S grounding in the image domain improves real-world occluded manipulation success rates by 26.7 percentage points over the strongest implicit visuo-tactile baseline, suggesting its improved spatial reasoning and robustness to occlusion.

   @misc{luo2026rgbsimagealignedtactilesaliency,
     title = {RGB-S: Image-Aligned Tactile Saliency for Robust Dexterous Manipulation},
     author = {Luo, Shengcheng and Wu, Kefei and Zhou, Xiaoying and Li, Wanlin and Jiao, Ziyuan and Xiao, Chenxi},
     year = {2026},
     eprint = {2606.08765},
     archiveprefix = {arXiv},
     primaryclass = {cs.RO},
     url = {https://arxiv.org/abs/2606.08765},
   }

2025

1. ICRA 2025

   

   Human-Agent Joint Learning for Efficient Robot Manipulation Skill Acquisition

   Shengcheng Luo^*, Quanquan Peng^*, Jun Lv^*, Kaiwen Hong, Katherine Rose Driggs-Campbell, Cewu Lu, and Yong-Lu Li

   In International Conference on Robotics and Automation, 2025

   🔥 Best Paper on Human-Robot Interaction 🔥 Abs Bib PROJECT Video

   We have been selected as the best paper finalist for our paper in the International Conference on Robotics and Automation (ICRA) 2025 and the best paper of human-robot interaction track.

   Employing a teleoperation system for gathering demonstrations offers the potential for more efficient learning of robot manipulation. However, teleoperating a robot arm equipped with a dexterous hand or gripper, via a teleoperation system poses significant challenges due to its high dimensionality, complex motions, and differences in physiological structure.In this study, we introduce a novel system for joint learning between human operators and robots, that enables human operators to share control of a robot end-effector with a learned assistive agent, facilitating simultaneous human demonstration collection and robot manipulation teaching. In this setup, as data accumulates, the assistive agent gradually learns. Consequently, less human effort and attention are required, enhancing the efficiency of the data collection process. It also allows the human operator to adjust the control ratio to achieve a trade-off between manual and automated control.We conducted experiments in both simulated environments and physical real-world settings. Through user studies and quantitative evaluations, it is evident that the proposed system could enhance data collection efficiency and reduce the need for human adaptation while ensuring the collected data is of sufficient quality for downstream tasks.

   @inproceedings{luo2024humanagentjointlearningefficient,
     title = {Human-Agent Joint Learning for Efficient Robot Manipulation Skill Acquisition},
     author = {Luo, Shengcheng and Peng, Quanquan and Lv, Jun and Hong, Kaiwen and Driggs-Campbell, Katherine Rose and Lu, Cewu and Li, Yong-Lu},
     year = {2025},
     eprint = {2407.00299},
     booktitle = {International Conference on Robotics and Automation},
     archiveprefix = {arXiv},
     primaryclass = {cs.RO},
     url = {https://arxiv.org/abs/2407.00299},
   }

2024

1. ECCV 2024

   

   DISCO: Embodied Navigation and Interaction via Differentiable Scene Semantics and Dual-Level Control

   Xinyu Xu, Shengcheng Luo, Yanchao Yang, Yong-Lu Li, and Cewu Lu

   In European Conference on Computer Vision, 2024

   Bib

   @inproceedings{xu2024disco,
     title = {DISCO: Embodied Navigation and Interaction via Differentiable Scene Semantics and Dual-Level Control},
     author = {Xu, Xinyu and Luo, Shengcheng and Yang, Yanchao and Li, Yong-Lu and Lu, Cewu},
     booktitle = {European Conference on Computer Vision},
     pages = {108--125},
     year = {2024},
     organization = {Springer},
   }

2021

1. ACM MM 2021

   

   Syntropic Counterpoints: Metaphysics of The Machines

   Predrag K* Nikolic, Ruiyang* Liu, and Shengcheng Luo

   In Proceedings of the 29th ACM International Conference on Multimedia, 2021

   Bib

   @inproceedings{nikolic2021syntropic,
     title = {Syntropic Counterpoints: Metaphysics of The Machines},
     author = {Nikolic, Predrag K and Liu, Ruiyang and Luo, Shengcheng},
     booktitle = {Proceedings of the 29th ACM International Conference on Multimedia},
     pages = {1443--1445},
     year = {2021},
   }