Haptics-based robot teleoperation for soft object manipulation

Robot teleoperation in the construction industry faces significant challenges, particularly when manipulating soft, deformable objects. Traditional systems predominantly focus on rigid object manipulation, often overlooking the complexities associated with soft materials that change shape and consistency under force. This study introduces a novel haptic feedback system integrated with visual feedback to enhance teleoperation performance for soft object manipulation. The proposed system comprises a multimodule setup, including a soft object modeling module, haptic feedback interface, robot control module, and physical digital twin module, ensuring seamless communication and synchronization. A human-subject experiment (N=30) was performed to test the efficacy of the system. Results demonstrate that the integration of haptic and visual feedback (multifeedback condition) significantly improves task performance, time efficiency, and path accuracy, and reduces cognitive load compared with conditions using these feedback modes independently. Haptic feedback alone also produced substantial improvements over visual-only and control conditions, highlighting its critical role in conveying essential physical properties and interactions. The implications for robot teleoperation in construction are promising, indicating that it can facilitate higher precision in tasks and safer work environments, and potentially lower training costs and time. Despite the controlled experimental environment and homogeneous participant pool, the findings underscore the potential of multisensory feedback systems in real-world construction settings. Future research will explore more-varied conditions and additional sensory modalities to further enhance operator performance and system applicability.