Vision-based detection of unsafe worker guardrail climbing based on posture and instance segmentation data fusion

Currently, the incidence of accidents involving falls from height at construction sites caused by workers climbing guardrails is still high. Traditional unsafe behavior management mainly relies on a safety patrol of construction-site supervisors, which consumes considerable laborpower and time. There is still a critical need for an automated safety management method to identify unsafe guardrail climbing behavior. This study proposes a worker behavior identification method based on visual data fusion of a worker's surrounding environment and posture data. Videos of seven participants' guardrail climbing behavior through multiangle and multidistance cameras were analyzed to verify this method. By analyzing the environment and posture of the participants, three methods based on environment, posture, and fusion data were used to detect the stage of guardrail climbing action of the workers and compare them with the ground truth labeled by safety experts. The precision and recall of worker guardrail climbing behavior based on the fusion method were 82% and 83% respectively, which is better performance than that obtained using a single method. The data fusion-based method avoids the misjudgment generated by a single detection method and can identify the guardrail climbing behavior more accurately.