Zhou, S; Wang, H; Ng, S T; Li, D; Xie, S; Chen, K and Wang, W (2025) Building a construction law knowledge repository to enhance general-purpose large language model performance on domain question-answering: A case of China. Engineering, Construction and Architectural Management, 32(13), pp. 518-546. ISSN 0969-9988
Abstract
Purpose – Achieving smart question-answering (QA) for construction laws (CLs) holds significant promise in aiding domain professionals with legal inquiries. Existing studies of construction law question-answering (CLQA) rely on learning-based models, which require extensive training data and are limited to a narrow QA scope. Meanwhile, general-purpose large language models (GPLLMs) possess great potential for CLQA but fall short of domain-specific knowledge. This study aims to propose a data-driven and expertise-based approach to develop a construction law knowledge repository (CLKR) and validate its effectiveness in enhancing the CLQA performance of GPLLMs. Design/methodology/approach – This methodology includes (1) recognizing 702 candidate CL documents from 374, 992 official judgments, (2) building a CLKR with 387 filtered documents covering eight CL knowledge areas, (3) integrating CLKR and seven representative GPLLMs and (4) constructing a 2, 140-question CLQA dataset from Professional Construction Engineer Qualification Examinations (PCEQEs) during 2014–2023 to compare CLQA performance between seven pairs of GPLLMs with and without CLKR. Findings – The CLKR significantly enhances the CLQA performance of seven GPLLMs, yielding an impressive average accuracy increase of 21.1%, with individual improvements ranging from 9.9 to 44.9%. Furthermore, CLKR boosts the accuracy of single-answer questions by 14.9% and multiple-answer questions by 38.3%. Additionally, the accuracy enhancements across 8 CL knowledge areas are between 14.5 and 28.2%. Originality/value – This study proposes an approach of developing the external knowledge base of CLKR to empower GPLLMs, significantly expanding the scope of CLQA while bypassing the complex training of traditional learning-based models. Moreover, this study confirms the effectiveness of CLKR in augmenting GPLLM performance and offers a reusable CLQA test dataset as a benchmark.
| Item Type: | Article |
|---|---|
| Uncontrolled Keywords: | construction laws; knowledge repository; large language models; question-answering |
| Index terms: | effectiveness, construction law, engineer, qualification, judgment, dataset, China, documents, methodology, accuracy, large language model, knowledge base |
| Subjects: | performance management, professional development, data management, construction law, information systems, profession, dispute resolution, data science, educational resources, research methods, Geography |
| Topics: | Legal Issues, Quality Management, Education, Geographical Context, Engineering Principles, Digital Applications, Research Practice, Information Management, Roles and Professions |
| Descriptive scope: | 3 PCT |
N.B. Descriptive scope is a count of how many of the five facets of empirical research are indicated by the words used in title, abstract and keywords. It is not intended as a judgement on the research; merely a count of the kind of word we would expect to indicate Phenomenon, Concepts, Theoretical framing, Empirical techniques, Analytical techniques. If all five are present, then a code of “5 PCTEA” will indicate this. If you feel the coding for this record is questionable, we welcome discussion around the terms we matched or the way we categorized them. The facet you would expect may not be coded, or a facet may be coded inappropriately. This can also bear on a larger question, of which facets should be treated as defining in construction management research. Please get in touch, and we will look at it. More details here