Automatic vision-based dump truck productivity measurement based on deep-learning illumination enhancement for low-visibility harsh construction environment

Productivity assessment plays a key role in successful earthwork projects and is primarily achieved by monitoring key construction equipment such as excavators and dump trucks. Vision-based methods are widely adopted to analyze the productivity of construction equipment in earthwork projects. These methods are inexpensive and easy to implement and maintain. However, previous studies on vision-based productivity analysis of earthmoving equipment have predominantly demonstrated its effectiveness under favorable conditions characterized by stable lighting, areas devoid of shadows, and high-visibility environments free from significant occlusion by other objects or terrain. There is a significant illumination difference and limited visibility under harsh low-visibility conditions at earthwork construction sites, which makes it difficult to achieve reliable identification accuracy using these already developed methods. To address this problem, this study proposes an automatic vision-based dump truck productivity measurement method based on a deep learning illumination enhancement algorithm combined with transfer learning for low-visibility, harsh conditions at earthwork construction sites. This method uses an internet of things (IoT) system equipped with a camera to capture the image and a deep learning illumination enhancement algorithm, trainable deep hybrid network (TDHN), to enhance the image quality under low-light conditions at earthwork sites. Then, a deep convolutional neural network image recognition algorithm, ResNet50, was combined with a transfer learning technique to extract information from the image. Through the IoT, this processed information is sent to the earthwork platform to perform productivity analysis and make timely decisions regarding equipment allocation schemes. To validate the effectiveness of the proposed methodology, it was implemented in a real-time earthwork project. This study results show that image recognition accuracy of 99.6%, 95.67%, and 94.77% under normal lighting, low lighting, and extremely low lighting conditions, respectively. The dump truck recognition accuracy increased by 1.10%, 3.62%, and 21.19%, leading to a significant improvement in productivity measurement of 1.08%, 3.54%, and 20.71% for the above-mentioned lighting conditions, respectively.