publications

2026

1. arxiv

   

   CAAT: Contact-Aware Attention Scaling and Tactile Masking for Data-Efficient Contact-Rich Manipulation

   Jiaming Jiang^*, Yuzhe Huang^*, Hao Liang, Pei Lin, Shengcheng Luo, Fanrong Dong, Jiaping Wu, Chenxi Xiao^†, Wanlin Li^†, and Ziyuan Jiao^†

   2026

   Abs Bib PROJECT Code

   In contact-rich manipulation, visual observations primarily guide motion in free space, whereas tactile observations become particularly informative during contact. However, standard Transformer-based visuo-tactile policies typically rely on either token concatenation or learnable gating. These approaches lack explicit contact-aware priors, making it difficult to efficiently learn effective cross-modal representations from demonstrations. To address this limitation, we propose CAAT, a lightweight contact-aware framework that explicitly incorporates contact priors through attention scaling and dynamic tactile masking. Specifically, CAAT emphasizes visual information before contact and tactile information during contact. It also suppresses static background tokens by comparing the current tactile observation with a non-contact reference. CAAT can be integrated into commonly used Transformer-based policies without modifying their action decoders. In simulation, integrating CAAT with ACT improves the average success rate by 18.0 percentage points over direct visuo-tactile fusion and by 10.0 percentage points over gated fusion. In real-world experiments using a visuo-tactile UMI platform, CAAT achieves an average success rate of 60.0% across ACT, Diffusion Policy, and \pi_0, outperforming the strongest baseline by an average of 21.1 percentage points. These results demonstrate that explicit contact priors and dynamic tactile masking are effective in improving visuo-tactile policy learning and task performance of diverse policy architectures.

   @misc{jiang2026caat,
     title = {CAAT: Contact-Aware Attention Scaling and Tactile Masking for Data-Efficient Contact-Rich Manipulation},
     author = {Jiang, Jiaming and Huang, Yuzhe and Liang, Hao and Lin, Pei and Luo, Shengcheng and Dong, Fanrong and Wu, Jiaping and Xiao, Chenxi and Li, Wanlin and Jiao, Ziyuan},
     year = {2026},
     eprint = {2608.01102},
     archiveprefix = {arXiv},
     primaryclass = {cs.RO},
     url = {https://arxiv.org/abs/2608.01102},
   }

2. arxiv

   

   DexMani: Human-Derived Manipulability Guidance for Dexterous Rotation

   Xiaoyang Chen^*, Shengcheng Luo^*, Haoran Guo, Jiaming Jiang, Wanlin Li, Ziyuan Jiao^†, and Chenxi Xiao^†

   2026

   Abs Bib PROJECT

   Dexterous object rotation is a sequential contact problem: each support, release, and re-contact decision must both produce the desired object motion, and prepare the hand configuration for continued rotation. Existing reinforcement learning methods discover such movement patterns through trial and error on specific robotic hand embodiments, without explicitly accounting for how each contact transition affects the hand’s ability to sustain object rotation in subsequent steps. We introduce DexMani, a framework that transfers human demonstrations as contact-conditioned manipulability evolution. This prior captures how successful human contact transitions reshape the object-rotation directions available to the hand. DexMani then learns this manipulability evolution and uses it to guide downstream reinforcement learning, enabling rotation skills to be acquired across robot embodiments with distinct kinematics and active-contact configurations. Across the Shadow Hand, Allegro Hand, and XHand, DexMani achieves the highest success rates in every evaluated setting for both seen and unseen objects. DexMani reaches an average success rate of 57.5% on LEAP Hand, outperforming other baselines and producing smoother rotatory motions.

   @misc{chen2026dexmanihumanderivedmanipulabilityguidance,
     title = {DexMani: Human-Derived Manipulability Guidance for Dexterous Rotation},
     author = {Chen, Xiaoyang and Luo, Shengcheng and Guo, Haoran and Jiang, Jiaming and Li, Wanlin and Jiao, Ziyuan and Xiao, Chenxi},
     year = {2026},
     eprint = {2608.00554},
     archiveprefix = {arXiv},
     primaryclass = {cs.RO},
     url = {https://arxiv.org/abs/2608.00554},
   }

3. arxiv

   

   Blind Dexterous Grasping via Real2Sim2Real Tactile Policy Learning

   Shengcheng Luo^*, Xiyan Huang^*, Zhe Xu, Wanlin Li, Ziyuan Jiao^†, and Chenxi Xiao^†

   2026

   Abs Bib PROJECT

   Blind grasping with a dexterous hand is a crucial manipulation capability. Nevertheless, learning such tactile-only policies for real robots remains challenging due to the tactile sim-to-real gap and the limited expressiveness of sparse tactile signals. To bridge this gap, we propose a framework for tactile-only blind grasping that is deployable on a physical multi-fingered robotic hand. Our approach combines three key components. First, we introduce a Real2Sim tactile calibration pipeline that constructs a contact-calibrated digital-twin simulator capable of reproducing real tactile signals. Second, we improve the expressiveness of sparse tactile observations using a layout-aware tactile encoder, which incorporates sensor-geometry priors through self-supervised pretraining. Third, to improve generalization to unseen objects, we train object-specific reinforcement-learning experts in the calibrated simulator and aggregate their successful grasp trajectories into a tactile-conditioned Diffusion Policy. We evaluate our method on a physical LEAP Hand equipped with distributed tactile sensing across 10 seen and 10 unseen objects. The deployed policy achieves a 27% real-world grasp success rate across all 20 objects, without real-world grasping demonstrations or visual input. Simulation ablations show that layout-aware tactile pretraining improves grasping performance, while sensing-level evaluations confirm that Real2Sim calibration increases the consistency of tactile contact events between simulation and hardware. Together, these results suggest that contact-event calibration, geometry-aware tactile representation learning, and diffusion-based policy aggregation provide an effective path toward tactile-only blind grasping on real dexterous robotic hands.

   @misc{luo2026blinddexterousgraspingreal2sim2real,
     title = {Blind Dexterous Grasping via Real2Sim2Real Tactile Policy Learning},
     author = {Luo, Shengcheng and Huang, Xiyan and Xu, Zhe and Li, Wanlin and Jiao, Ziyuan and Xiao, Chenxi},
     year = {2026},
     eprint = {2606.11767},
     archiveprefix = {arXiv},
     primaryclass = {cs.RO},
     url = {https://arxiv.org/abs/2606.11767},
   }

4. arxiv

   

   RGB-S: Image-Aligned Tactile Saliency for Robust Dexterous Manipulation

   Shengcheng Luo^*, Kefei Wu^*, Xiaoying Zhou, Wanlin Li, Ziyuan Jiao^†, and Chenxi Xiao^†

   2026

   Abs Bib PROJECT

   Effective visuo-tactile integration is critical for robotic dexterous manipulation, especially when visual observations are unreliable or occluded. However, robustly aligning sparse, heterogeneous tactile measurements with dense visual representations remains a fundamental challenge. Most existing approaches require policies to learn cross-modal correspondences implicitly from limited demonstrations, without leveraging geometric priors. As a result, they are often data-inefficient and generalize poorly when visual observations are degraded. To address this limitation, we propose a framework that explicitly grounds physical contacts in the image domain. Using robot forward kinematics and camera calibration, we project tactile sensor locations directly onto the RGB image plane. We then render force-modulated Gaussian saliency maps to model spatial uncertainty arising from kinematic and calibration errors. By integrating these 2D spatial anchors through a zero-initialized conditioning architecture, our method injects physical contact priors into standard visual backbones while preserving pre-trained visual representations. We evaluate our method on six dexterous manipulation tasks in both simulation and the real world under severe visual occlusions. Real-world experiments show that explicit RGB-S grounding in the image domain improves real-world occluded manipulation success rates by 26.7 percentage points over the strongest implicit visuo-tactile baseline, suggesting its improved spatial reasoning and robustness to occlusion.

   @misc{luo2026rgbsimagealignedtactilesaliency,
     title = {RGB-S: Image-Aligned Tactile Saliency for Robust Dexterous Manipulation},
     author = {Luo, Shengcheng and Wu, Kefei and Zhou, Xiaoying and Li, Wanlin and Jiao, Ziyuan and Xiao, Chenxi},
     year = {2026},
     eprint = {2606.08765},
     archiveprefix = {arXiv},
     primaryclass = {cs.RO},
     url = {https://arxiv.org/abs/2606.08765},
   }

2025

1. ICRA 2025

   

   Human-Agent Joint Learning for Efficient Robot Manipulation Skill Acquisition

   Shengcheng Luo^*, Quanquan Peng^*, Jun Lv^*, Kaiwen Hong, Katherine Rose Driggs-Campbell, Cewu Lu, and Yong-Lu Li

   In International Conference on Robotics and Automation, 2025

   🔥 Best Paper on Human-Robot Interaction 🔥 Abs Bib PROJECT Video

   We have been selected as the best paper finalist for our paper in the International Conference on Robotics and Automation (ICRA) 2025 and the best paper of human-robot interaction track.

   Employing a teleoperation system for gathering demonstrations offers the potential for more efficient learning of robot manipulation. However, teleoperating a robot arm equipped with a dexterous hand or gripper, via a teleoperation system poses significant challenges due to its high dimensionality, complex motions, and differences in physiological structure.In this study, we introduce a novel system for joint learning between human operators and robots, that enables human operators to share control of a robot end-effector with a learned assistive agent, facilitating simultaneous human demonstration collection and robot manipulation teaching. In this setup, as data accumulates, the assistive agent gradually learns. Consequently, less human effort and attention are required, enhancing the efficiency of the data collection process. It also allows the human operator to adjust the control ratio to achieve a trade-off between manual and automated control.We conducted experiments in both simulated environments and physical real-world settings. Through user studies and quantitative evaluations, it is evident that the proposed system could enhance data collection efficiency and reduce the need for human adaptation while ensuring the collected data is of sufficient quality for downstream tasks.

   @inproceedings{luo2024humanagentjointlearningefficient,
     title = {Human-Agent Joint Learning for Efficient Robot Manipulation Skill Acquisition},
     author = {Luo, Shengcheng and Peng, Quanquan and Lv, Jun and Hong, Kaiwen and Driggs-Campbell, Katherine Rose and Lu, Cewu and Li, Yong-Lu},
     year = {2025},
     eprint = {2407.00299},
     booktitle = {International Conference on Robotics and Automation},
     archiveprefix = {arXiv},
     primaryclass = {cs.RO},
     url = {https://arxiv.org/abs/2407.00299},
   }

2024

1. ECCV 2024

   

   DISCO: Embodied Navigation and Interaction via Differentiable Scene Semantics and Dual-Level Control

   Xinyu Xu, Shengcheng Luo, Yanchao Yang, Yong-Lu Li, and Cewu Lu

   In European Conference on Computer Vision, 2024

   Bib

   @inproceedings{xu2024disco,
     title = {DISCO: Embodied Navigation and Interaction via Differentiable Scene Semantics and Dual-Level Control},
     author = {Xu, Xinyu and Luo, Shengcheng and Yang, Yanchao and Li, Yong-Lu and Lu, Cewu},
     booktitle = {European Conference on Computer Vision},
     pages = {108--125},
     year = {2024},
     organization = {Springer},
   }

2021

1. ACM MM 2021

   

   Syntropic Counterpoints: Metaphysics of The Machines

   Predrag K* Nikolic, Ruiyang* Liu, and Shengcheng Luo

   In Proceedings of the 29th ACM International Conference on Multimedia, 2021

   Bib

   @inproceedings{nikolic2021syntropic,
     title = {Syntropic Counterpoints: Metaphysics of The Machines},
     author = {Nikolic, Predrag K and Liu, Ruiyang and Luo, Shengcheng},
     booktitle = {Proceedings of the 29th ACM International Conference on Multimedia},
     pages = {1443--1445},
     year = {2021},
   }