Wang, Yiheng (2023) Vision-assisted behavior-based construction safety: Integrating computer vision and natural language processing. PhD thesis, University of Alberta, Canada.
Abstract
Abstract: Background: Construction sites can be hazardous places. Behavior-based safety is a method to optimize workers' behaviors and improve site safety. Previous behavior-based safety has been criticized for their low efficiency because of manual observation. The community has conducted enormous studies about applying advanced computer vision-based methods to automate the monitoring and observation of construction sites. However, the lack of methods for extracting semantic information and identifying safety hazards from construction imagery still poses a significant challenge for the development of sophisticated vision-assisted behavior-based safety programs. Objectives: This research aims to automate the processes where the manual observation and inspection is needed in the traditional construction safety management by (1) enrich the information could be extracted from construction images, supporting safety hazard identification, (2) automate the safety hazard identification on site, and enable reasoning about the hazard identification according to safety regulations, and (3) automate the image records management and retrieval for efficient safety analysis. Methods: Firstly, this research proposes a method to extract objects, activities, and interaction information from construction images. This method utilizes image captioning techniques to generate image captions for construction images containing semantic information. Secondly, this research proposes a novel visual–text semantic similarity method to compare construction image captions with safety regulation rules, enabling automatic safety hazard identification and reasoning. Finally, this research proposed a novel content-based image retrieval method for construction image repositories-based object detection. This will help safety managers query and retrieve similar cases from monitoring image records, and conduct behavior analysis. Outcomes: This research will improve the current vision-based construction management applications in the following ways: (1) it helps automate the monitoring and observation of construction sites; (2) it provides an automated method to identify potential safety hazards on construction sites and give reasoning of their violation about safety rules; and (3) it provides an information retrieval system for construction image repositories, enabling fast image retrieval and case-based reasoning and analysis.
| Item Type: | Thesis (Doctoral) |
|---|---|
| Thesis advisor: | Al-Hussein, Mohamed and Bouferguene, Ahmed |
| Uncontrolled Keywords: | construction safety; deep learning; computer vision; natural language processing; behavior-based safety |
| Index terms: | regulation, monitoring, deep learning, computer vision, hazard identification, inspection, object detection, efficiency, reasoning, information retrieval, safety programs, violation, interaction, case-based reasoning, construction site, construction safety, traditional construction, manager, safety management |
| Subjects: | data management, environmental health, performance management, behavioral psychology, computer vision, control systems, occupational health and safety management, practitioner, financial risk, political science, heritage and conservation, regulatory law, artificial intelligence, work location, quality assurance, cognitive psychology |
| Topics: | Site Management, Research Practice, Governance, Sustainability, Legal Issues, Roles and Professions, Cost Management, Design Practice, Quality Management, Digital Applications, Health and Safety |
| Descriptive scope: | 3 PCT |
N.B. Descriptive scope is a count of how many of the five facets of empirical research are indicated by the words used in title, abstract and keywords. It is not intended as a judgement on the research; merely a count of the kind of word we would expect to indicate Phenomenon, Concepts, Theoretical framing, Empirical techniques, Analytical techniques. If all five are present, then a code of “5 PCTEA” will indicate this. If you feel the coding for this record is questionable, we welcome discussion around the terms we matched or the way we categorized them. The facet you would expect may not be coded, or a facet may be coded inappropriately. This can also bear on a larger question, of which facets should be treated as defining in construction management research. Please get in touch, and we will look at it. More details here