Embodied robot teleoperation in construction

Human-robot collaboration is a key component of Industry 4.0. In the construction industry, collaborative robots have been utilized to increase human workers capabilities and facilitate complex construction operations for improved productivity and safety. Robot teleoperation, i.e., the manipulation of remote robotic systems from a distance, has gained popularity in a variety of construction applications. However, the human-robot interaction (HRI) designs for construction teleoperation are hindered by the complexity and diversity of operational requirements, such as the evolving work environment and unpredictable workflows. Traditional HRI for robot teleoperation often depends on visual data (e.g., video streaming), which results in a restricted field of view (FOV) and an increased cognitive load for processing the extra spatial information. The gap between robotic perception and human sensory processes makes it challenging for operators to acquire embodied cognition, such as estimating the force needed for an action, and the distance between a remote robot and obstacles. This research aims to design and examine an innovative embodied robot teleoperation interface based on sensory augmentation for construction applications. The proposed system relies on mixed reality (MR) and physics engine simulations to transfer the realistic physical interactions of the remote robotic system to the human operator. Sensor data from the remote robot is processed and enhanced with simulation. Then a sensory augmentation system reproduces both visual and haptic cues from the remote robot, via the Unity game engine, and manifested via VR devices and haptic devices. With this proposed design, the perception of remote robot status and physical features of workspaces can be transferred to the human operator in a more immersive and embodied way. Three studies will be conducted to design and evaluate the visual and haptic feedback methods for robot teleoperation in order to determine how the proposed sensory augmentation designs improve HRI in construction-specific tasks. Then a prediction-based method is proposed to improve the rendering efficiency by predicting human motion intents. Finally, a human-subject experiment will be performed to validate the efficacy of the proposed system. The system and operational performance as well as motor function changes will be collected and analyzed via human and task evaluations. The contribution of this proposed research is expected to generate new frameworks and knowledge about embodied robot teleoperation for dexterity-driven construction tasks. The findings can also provide practical design and training guidelines for conveying visual and haptic information to human operators for future robot teleoperation in construction tasks.