Accounting for rater credibility when evaluating construction industry service providers

This study investigates how source credibility theory can support reputation mechanisms in AEC electronic commerce. Researchers and commercial interests have developed rating mechanisms that support trust in primarily consumer-to-consumer electronic market places. In contrast to consumer electronic marketplaces, the raters in business-to-business communities are skilled and connected, necessitating a reputation mechanism to account for the relationship between the user and the rater. Source credibility theory is an area of communication science that explicitly studies and formalizes trust between human actors. A rating system based on source credibility offers several advantages over existing models including tested frameworks for aggregating ratings from different sources and validated scales for measuring a source's (rater's) credibility. In addition, the weights of a rater's ratings depend on user preferences instead on rater behavior, which decreases the amount of data required to calibrate the model. I have divided the fundamental research question: How can source credibility theory support rating systems in the procurement of AEC services?, into the two dimensions: operationalization and added value. To investigate the research question, I operationalized source credibility into a credibility-weighted rating model, which assigns weights based on rater credibility. Furthermore, in two experiments, a set of industry users applied a credibility-weighted tool and an unweighted tool to evaluate bids from AEC subcontractors. Both experiments showed with statistical significance that the credibility-weighted models predicted rater weights better than an unweighted model. This study therefore contributes a methodology to operationalize source credibility theory to calculate rater weights for AEC. The experiments also showed that industry practitioners varied their evaluations more, and also were more confident in their judgments, when using a credibility-weighted tool than when using an unweighted tool. This study therefore provides evidence that a credibility weighted rating tool adds value in the process of evaluating AEC subcontractors by increasing the decision-maker's confidence in the accuracy of the information provided by the rating tool. I claim that these findings have power and generality and contribute to the literature of AEC electronic commerce. AEC Bidding, reputation mechanisms in electronic commerce, and applicability of source credibility theory.