Tebi-llama: Temporal binding foundation model for material property recognition via tactile sequence perception

Robotic systems operating in complex construction environments require detailed material property recognition to perform tasks such as handling deformable objects, assembling components, and monitoring structure integrity. While vision-based methods have traditionally dominated perception, they often face significant limitations in cluttered, low-visibility settings or when subtle contact-based cues are essential. This paper presents TeBi-Llama, a temporal binding foundation model that fuses sequential tactile data from GelSight sensors for enhanced material classification and descriptive interpretation. Leveraging long short-term memory modules to capture evolving contact interactions, the proposed approach integrates these sequential features into a large-scale language model (LLaMA-2) via a structure-aware fusion mechanism. Experimental results demonstrate that TeBi-Llama improves top-1 classification accuracy by 11% and top-5 accuracy by 17% over static single-frame baselines, while also producing richer and more accurate open-ended material descriptions. These findings highlight the critical role of temporal tactile sensing and advanced multimodal integration for next-generation construction robotics, potentially supporting safer, more adaptive, and efficient automated operations on dynamic, unstructured job sites.