Video-based human motion capture and force estimation for comprehensive on-site ergonomic risk assessment

Construction is one of the most hazardous industries with high non-fatal injuries because it involves physically demanding tasks performed in an unstructured and dynamic environment. Work-related musculoskeletal disorders (WMSDs) are the major cause of non-fatal injuries. Various methods, such as self-report, observation, and direct measurement, are used for assessing the risk level of WMSDs by quantifying the ergonomic risk factors (e.g., posture, repetition, and force). However, they are either time consuming or error-prone (e.g., self-report and observation) or invasive (e.g., direct measurement). The recent advancement of computer vision allows for rapid, accurate, and non-invasive motion capture only using ordinary cameras. Key challenges remain for applying to assess jobs’ ergonomic risks: 1) long-lasting occlusion in a construction site creates an obstacle to enforcing kinematic and temporal consistency between frames to estimate posture’s frequency and repetition; 2) as a critical risk factor for ergonomic risk assessment, force is very challenging to non-invasively estimate, which hinders field applications; and 3) little effort has been made for comprehensive ergonomic risk assessment. These knowledge gaps were addressed by three research objectives: 1) develop and validate a video-based human motion capture framework to quantify ergonomic risk factors of posture and its repetition by extracting continuous 2D/3D human model with enforced kinematic and temporal consistency; 2) develop and validate a video-based hand push force estimation framework; and 3) apply the risk factors estimated by videos to comprehensive ergonomic risk assessment tools including postural and biomechanical analysis. Results yielded around 11.6 and 7.5 degrees of joint angle estimation error for 2D and 3D motion captures, respectively, despite prevalent occlusions. Also, resultant frequency and duration comparison with experienced ergonomists’ observation demonstrates a great potential to robustly quantify jobs’ ergonomic risk factors of posture and repetition. Lab-based testing shows an accurate peak force occurrence time and peak force magnitude estimation, suggesting a potential to quantify critical variables of push force exertion only from videos. By applying the collected risk factors comprehensively to several ergonomic risk assessment tools, it demonstrates a promising level of risk assessment accuracy compared with expert observation and sensor-based measurement. The proposed video-based motion capture and force estimation frameworks for comprehensive ergonomic risk assessment are expected to greatly reduce the time and effort of on-site data collection and increase the number of evaluated jobs with higher frequency thereby providing a better opportunity to understand and control WMSDs.