This article was downloaded by: [HEAL-Link Consortium] On: 4 January 2011 Access details: Access Details: [subscription

number 786636552] Publisher Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 3741 Mortimer Street, London W1T 3JH, UK

Construction Management and Economics

Publication details, including instructions for authors and subscription information: http://www.informaworld.com/smpp/title~content=t713664979

The accuracy of pre-tender building cost estimates in Australia

Ajibade Ayodeji Aibinua; Thomas Pascoa a Faculty of Architecture, Building and Planning, University of Melbourne, Parkville, Melbourne, 3010 Australia

To cite this Article Aibinu, Ajibade Ayodeji and Pasco, Thomas(2008) 'The accuracy of pre-tender building cost estimates

in Australia', Construction Management and Economics, 26: 12, 1257 1269 To link to this Article: DOI: 10.1080/01446190802527514 URL: http://dx.doi.org/10.1080/01446190802527514

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf This article may be used for research, teaching and private study purposes. Any substantial or systematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.

Construction Management and Economics (December 2008) 26, 12571269

The accuracy of pre-tender building cost estimates in Australia

AJIBADE AYODEJI AIBINU* and THOMAS PASCO
Faculty of Architecture, Building and Planning, University of Melbourne, Parkville, Melbourne, 3010 Australia Received 22 January 2008; accepted 4 October 2008

Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011

A pre-tender building cost estimate is an important piece of information when making decisions at the project planning and design stage. The important project characteristics influencing the accuracy of pre-tender building cost estimates are examined and practical improvement for increasing the accuracy of estimates are considered. A quantitative approach is used to address the research problem. Analysis of data from 56 projects and from a postal questionnaire survey of 102 quantity surveying firms suggests that the accuracy of pre-tender building cost estimates varies according to project size and principal structural material. When eight identified project characteristics are controlled in a multiple regression analysis, the accuracy of estimates is influenced by project size. The estimates of smaller projects are more biased than the estimates of larger projects. It was discovered that pre-tender building costs are more often overestimated than are underestimated. Overestimated forecasts are incorrect by a larger amount than underestimated forecasts. Data analysis also revealed that the accuracy of pre-tender building cost estimates has not improved over time. The majority of the respondents are somewhat dissatisfied with the accuracy of estimates in the industry. Probability estimation and simulation of past estimates, reducing quantity surveying and cost engineering skill turnover, incorporating market sentiments into estimates, early involvement of the quantity surveyor at the brief stage, and proper documentation of experience gained in the estimation of projects should help firms increase the accuracy of estimates for new projects. Keywords: Australia, estimating accuracy, pre-tender estimates, quantity surveying, tendering.

Introduction
Pre-tender building cost estimates are susceptible to inaccuracies (bias) because they are often prepared within a limited timeframe, and without finalized project scope. Pursuing an underestimated project can lead to project failure. On the other hand, overestimation of a project at the pre-tender stage can lead to a viable project being dropped or re-tendered when there is no bid close enough to permit project award. Bias in the estimate of a project may arise from two sources, namely, bias associated with the project itself (will be the same regardless of the estimator) and bias associated with the estimating technique used and the environment (which would change depending on the estimator). The only known published works relating to accuracy of cost forecasts in an Australian

context are Mills (1997), which compared prediction of building price movement by quantity surveyors with the Australian Bureau of Statistics actual building price movement, and Bromilow et al. (1988), which analysed the variance between contract sum and final contract sum. The objectives of this study are: (1) to explore the frequency and size of inaccuracy in pre-tender building cost estimates (i.e. the variance between pre-tender cost estimate and contract sumaccepted tender) using Australian data; to explore project characteristics influencing the accuracy of pre-tender building cost estimates; to assess whether the accuracy of pre-tender building cost estimates has improved over time; to investigate what firms are doing to improve the accuracy of cost estimates in practice, and in that regard, evaluate the effectiveness of the improvement methods.

(2) (3) (4)

*Author for correspondence. E-mail: aaibinu@unimelb.edu.au

Construction Management and Economics ISSN 0144-6193 print/ISSN 1466-433X online # 2008 Taylor & Francis http://www.tandf.co.uk/journals DOI: 10.1080/01446190802527514

1258
It is important to study the accuracy of building cost estimates because a large part of a quantity surveyors and cost engineers role in the construction industry is to provide certainty of cost to clients. Thus the knowledge developed in this study should help quantity surveyors, estimators and cost engineers so that they are aware of how their cost forecasts have performed over time and what project characteristics need special attention during cost estimation; and what practices could increase the accuracy of their estimates.

Aibinu and Pasco

the world, Trost and Oberlender (2003) identified 45 factors contributing to the accuracy of early stage estimates. They summarized the factors into 11 orthogonal elements. Of the 11 factors, the five most important include: process design, team experience and cost information, time allowed to prepare estimates, site requirements, and bidding and labour climate. All these studies suggest that there are a large number of variables that may substantially influence the accuracy of an early stage estimate. According to Gunner (1997) the factors influencing accuracy of estimates are intercorrelated so that the true bias of one factor could be masked by one or more factors. For example, Gunner and Skitmore (1999b) theorize that Price Intensity alone is both necessary and sufficient to account for systematic bias (inaccuracy) in building price forecasting. Price intensity is the total cost of a building divided by the gross floor area. Price intensity theory states that buildings with low unit rates (cost/m2 gross floor area) would tend to be overestimated, while those with high unit rates would tend to be underestimated. In a study of 89 construction projects in Hong Kong, Skitmore and Drew (2003) support the price intensity theory. In another study, Skitmore and Picken (2000) using data from 217 projects in the United States found that year was the underlying variable responsible for the bias and inconsistency in cost estimates, after partialling out confounding effects of the four factors put forward. The finding contrasts Gunner and Skitmores (1999b) price intensity theory. However, their result supports Gunners (1997) theory which states that intercorrelations among variables cause confounding effects. It also supports Gunner and Skitmore (1999a) in their suggestion that a single underlying variable is the cause of bias and consistency seen in estimates. Study hypotheses Based on a review of past studies, the following hypotheses are set out:
Hypothesis 1: Systematic bias and inconsistency in pretender building cost estimates are influenced by project size (measured by project value, number of storeys and gross floor area), location, project type, procurement route, project sector, price intensity and principal structural material. Hypothesis 2: The accuracy of pre-tender building cost estimates has not improved over time. Hypothesis 3: Quantity surveying firms agree on the most effective ways they believe will improve the accuracy of pre-tender building cost estimates.

Theoretical framework and scope of study

What is accuracy of an estimate? Pre-tender cost estimation (or early stage cost estimation) is the forecasting of the cost of a project during the planning and design stage (Serpell, 2005). Skitmore (1991) describes the accuracy of early stage estimation as comprising two aspects, namely, bias and consistency of the estimate when compared with the contract or accepted tender price. Bias is concerned with the average of differences between actual tender price and forecast while consistency of estimates is concerned with the degree of variation around the average. Factors affecting estimating accuracy An overview of previous studies suggests that a large number of factors may influence the accuracy of an estimate. Gunner and Skitmore (1999a) reviewed previous studies and summarized the factors as follows: building function, type of contract, conditions of contract, contract sum, price intensity, contract period, number of bidders, good/bad years, procurement basis, project sector (public, private or joint), number of priced items and number of drawings. Gunner and Skitmore (1999a) analysed the estimates of 181 projects in Singapore. They found that a majority of the factors influenced the accuracy of estimates. Using data from 42 projects in Singapore Ling and Boo (2001) found similar results when they compared five variables against Gunner and Skitmores (1999a) work. Skitmore and Picken (2000) studied the effect that four independent factors (building type, project size, project sector and year) had on estimating accuracy. They tested the four factors using data from 217 projects in the United States of America. They found that bias in the estimate of the projects is influenced by project size and year, while consistency in the estimates is influenced by project type, size and year. In a study of 67 process industry construction projects around

Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011

Building cost estimates Research method

Data collection Two methods of data collection were used to achieve the study objectives. In order to address objectives 1, 2 and 3, data were collected from the files of 56 construction projects completed between 1999 and 2007. The data were obtained from the office of a quantity surveying firm in Australia. Information obtained in respect of each project included: project value, the number of floors, gross floor area (GFA), project location (central business district, metropolitan, or rural), project type (residential, industrial or commercial), procurement route used, project sector (public, private or joint), price intensity (measured by ratio of project value and gross floor area), and principal structural material used (steel, concrete or timber). The projects were selected by a simple random sampling process. A list of projects completed from 1999 to 2007 was drawn. Projects that were not suitable for analysis, owing to incorrect job type or lack of early stage estimate, were discarded. Thereafter, each project was assigned a serial number sequentially from one. Random numbers were then generated using the Microsoft Excel program. The process yielded 85 random numbers. Numbers that were repeated were deleted the second time they appeared. The process produced 56 random numbers. The projects with serial numbers corresponding with the 56 random numbers

1259
were selected for data collection and analysis. As the researchers were allowed first-hand access to all data, no data were selected on the recommendation of the quantity surveyor responsible for the estimates. Thus there was no bias in the data collection. In order to achieve objective 4, a structured questionnaire was designed for data collection. It comprised questions regarding the profile of the respondents, the profile of their company, the respondents satisfaction with the current level of estimate accuracy in the industry, and the views of the respondents regarding acceptable level of estimate accuracy. The respondents were also asked to rate 12 methods that could be used for improving the accuracy of estimates. Depending on the nature of the question, respondents were asked to indicate their answers on a five-point Likert scale or a categorical scale. The questionnaires were mailed to 102 quantity surveying firms in July/August 2007. The firms were randomly selected from the list of about 166 firms maintained by the Australian Institute of Quantity Surveyors (AIQS) (AIQS, 2003).

Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011

Data analysis, results and discussion

Response rate and characteristic of sample from questionnaire The questionnaire survey yielded a response rate of 41%. Figure 1 shows the geographical distribution of

Figure 1 Geographical distribution of the survey and responses Key ACT Act Capital Territory; NSW New South Wales; NT Northern Territory; QLD Queensland; VIC Victoria; SA South Australia; TAS Tasmania; WA Western Australia

1260
the responses. 29% of the responding firms have fewer than 6 technical staff, 42% have between 6 and 15 technical staff, 22% have between 16 and 25 staff, and 7% have 26 or more technical staff. 90% of the questionnaires were completed by either a director or an associate of the firms while 10% were completed by a senior quantity surveyor or a quantity surveyor. 81% of them have had over 15 years experience, 13% have between 10 and 15 years experience, and 6% have fewer than 10 years experience. Altogether, the data have come from the highest echelon of quantity surveying professionals in Australia, and their responses can be confidently relied upon. Respondents tolerance and satisfaction with estimate accuracy in the industry
Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011

Aibinu and Pasco

of the impact of project sector. There were too few samples of projects procured by the private sector. There were also too few samples of projects procured jointly by the public and private sectors. Frequency and size of inaccuracies in pre-tender building cost forecasts Percentage cost overestimate or underestimate (estimate error or bias) were estimated for each project by using the following expression: Estimate bias~ pre tender cost estimate{accepted tender sum |100 accepted tender sum The mean estimate bias was also computed for the 56 projects using the following expression: P x Mean estimate biasx~ n where x5estimate bias; n5number of projects. A positive value of estimate bias implies an overestimation of cost while a negative value implies an underestimation of cost. The analysis shows that in about 7 out of every 10 projects, cost forecasts were overestimated while in about 3 out of every 10 projects cost forecasts were underestimated. This implies that pre-tender cost forecasts were more often overestimated than were underestimated. Bias in overestimated costs ranges from +0.97% to +31.88% with a mean of +10% while bias in underestimated cost ranges from 22.21% to 219.83% with a mean of 29%. A one-sample t-test (Levine et al., 2005) was used to test the following hypotheses: (1) The mean bias in overestimated costs (+10%) is not different from zero; (2) The mean bias in underestimated costs (29%) is not different from zero. The results show that the mean bias in overestimated costs (+10%) is significantly different from 0% (p50.000, standard deviation57%) which implies that the overestimated forecasts are truly biased and not by chance. Similarly, the mean bias in underestimated costs (29%) is significantly different from zero (p50.000, standard deviation55%) which also implies that underestimated forecasts are truly biased and not by chance. Analysis shows that the cost estimates for the 56 projects are generally biased and were overestimated (mean524.29%) (see Table 1). Again, one-sample t-test analysis shows that the mean (+4.29%) is significantly different from zero (p50.004, standard deviation510.61%) meaning that the estimates are biased overall. However, when one considers the fact that early stage estimates are prepared with little information, the overall mean estimate bias of 24.29% may be acceptable.

None of the respondents indicated that they were very satisfied with the current level of estimate accuracy. 66% indicated they were very dissatisfied, dissatisfied or neither satisfied nor dissatisfied, while 34% indicated that they were satisfied. When early stage estimate is compared with the lowest tender (contract sum), 24% of respondents believed that a tolerance limit of within 5% is acceptable, 54% nominated a tolerance of 10%, 20% nominated a tolerance of 20%, and 2% nominated a tolerance of 30%. Also, 70% of respondents indicated that the accuracy of early stage estimates hadnt improved at all, very slightly or slightly over the past 15 years while 25% believed that it has improved almost adequately or adequately. 5% did not respond.

Preliminary analysis of data from past projects

Treatment of data Data obtained from the 56 projects were analysed by project size (measured by project value, gross floor area and number of storeys), location, procurement route, project type, principal structural material and price intensity. Time of estimate was controlled by transforming the pre-tender estimate and contract sum of each project to the December 2006 price using the building price index published by Rawlinsons (2006, 2007). Also, differences in estimating processes and approach were understood to have been controlled because projects analysed were undertaken in the same company under the same quality assurance procedure. Project sector (whether private or public or joint) was also excluded from the analysis because the 56 projects did not provide a large enough spread of data to enable statistical analysis

Building cost estimates

Table 1 Preliminary analysis of 56 projects Number of projects 20 16 20 19 19 18 29 17 10 10 34 12 38 18 18 9 29 9 14 29 20 18 18 56 Mean error (%) (estimate bias) 8.95% 1.38% 1.95% 5.32% 5.68% 1.72% 9.31% 21.59% 20.30% 20.90% 4.82% 7.08% 2.66% 7.72% 2.17% 8.11% 4.41% 9.78% 7.29% 0.69% 7.55% 3.00% 1.94% 4.29% Standard deviation (%) 10.85% 7.79% 11.16% 12.32% 8.54% 10.76% 9.32% 8.76% 10.53% 12.71% 9.42% 11.41% 11.22% 8.48% 7.84% 10.93% 11.92% 10.84% 7.41% 10.54% 8.63% 11.54% 11.30% 10.61%

1261

Project factors broken into sub-groups *Project value ($) 15 000 000 5 000 00110 000 000 10 000 000+ **GFA (m2) 13000 300110 000 Above 10 000 Number of storeys 12 storeys 37 storeys 8+ storeys Location ***CBD Metropolitan Rural Procurement route Traditional (lump sum) Design & construct Project type Residential Industrial Commercial Principal structural material Timber Steel Concrete Price intensity ($/m2 GFA) 11200 12011700 Above 1700 Total

Coefficient of variation (%) (estimate consistency) 9.96% 7.68% 10.95% 11.70% 8.08% 10.58% 8.53% 8.90% 10.56% 12.83% 8.99% 10.66% 10.93% 7.87% 7.67% 10.11% 11.42% 9.87% 6.90% 10.47% 8.02% 11.20% 11.08% 10.17%

Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011

Notes: *Project value is measured in Australian dollars ($); **GFA5gross floor area; ***CBD5central business district.

In order to examine whether the error of underestimation is the same as the error of overestimation, the MannWhitney test (non-parametric) (Levine et al, 2005) was used instead of a two-sample t-test because normality of data could not be demonstrated. The test shows that the mean bias in underestimated costs (29%) is not the same as the mean bias in overestimated costs (+10%) (they are statistically and significantly different: p50.000). This means that pre-tender cost forecasts that were overestimated are incorrect by a larger margin than pre-tender cost forecasts that were underestimated. Standard deviation (S) was computed for the 56 projects using the expression: r . P x{x2 S~ n where x5estimate bias; n5number of projects. x5mean estimate bias;

Thereafter, the consistency in the estimates was determined by calculating the coefficient of variation (CV) as follows: CV~ standard deviation |100 mean estimate error

The standard deviation and CV were also determined for projects in the different groups of the eight factors examined (Table 2). Coefficient of variation is a measure of predictability of estimate bias. Large coefficient of variation implies that estimate bias is volatile and unpredictable. By visual inspection of the results (Table 2), the estimates for the 56 projects (put together) are inconsistent with a coefficient of variation of 10.17%. It is assumed that a double digit coefficient of variation is large. Thus the risk of estimation bias is not small. It also suggests that firms have little control over the propensity that estimates would be biased.

1262
Table 2 Result of ANOVA and Levenes test for homogeneity of variance* ANOVA (for estimate bias) Project factor Project value Gross floor area No. of storeys Project location Procurement route Project type Structural material Price intensity F 3.27 0.77 11.86 1.70 2.88 0.94 3.96 1.55 R 0.046* 0.466 0.001* 0.193 0.096 0.395 0.026* 0.222

Aibinu and Pasco

Levenes test (for estimate consistency) Test statistic 1.006 1.202 1.387 0.123 0.577 0.921 0.677 0.729 R 0.372 0.309 0.244 0.885 0.451 0.404 0.570 0.487

Notes: *Significant relationships taken at the 5% level.

Projects estimate bias and consistency across project factors Estimate bias and estimate consistency for each group in the eight project factors were examined. The mean estimate bias and coefficient variation (estimate consistency) were also determined for the projects in each group of the factors shown in Table 1. The results (Table 1) show that estimates of projects in the three project value categories tend to be overestimated. The estimates of projects with lower project value are more biased than estimates of projects with larger value. The same trend is observed for estimates of projects in the different GFA and number of storeys categories. Thus the estimates of smaller projects were more biased than the estimates of larger projects. The result for GFA is consistent with Skitmore and Drew (2003) and Ling and Boo (2001) which both found that the estimates of projects of smaller gross floor area tends to be more biased than the estimates of projects of larger gross floor area. However, for project value, the result is inconsistent with Skitmore and Drew (2003) where estimates of projects with a smaller value (less than $60m) are less biased than estimates of projects with a higher value (over $60m). Further, the cost estimates for traditionally procured projects tend to be more accurate than the cost estimates of those procured using design and construct method. Cost estimates of residential projects tend to be the least biased, followed by estimates of commercial and industrial projects. The findings could be adduced to experience; for example, the quantity surveying firm from which the sample projects were obtained has handled many traditionally procured projects and residential and commercial projects. This could be the reason why the estimates for those projects are less biased. Projects that were procured with design and construct method, as well as industrial projects are less common in the firms experience. This could explain why estimates for those projects are more biased.

The preliminary analysis (Table 1) also suggests that cost estimates of projects using concrete as the principal structural material are by far the least biased (overestimated by 0.7%), followed by steel with an average overestimate of 7.2%. Estimates of projects using timber as their primary structural material tend to be the most biased, with an average overestimate of 9.8%. Again, this result could be explained by experience. The sample projects indicate that the firm from which the data were drawn has handled fewer projects using timber when compared to projects constructed with steel and concrete. With regard to price intensity, the trend seems to be negatively related. Estimates of projects with low $/m2 GFA tend to be more biased than estimates of projects with high $/m2 GFA. This is contrary to Gunner and Skitmores (1999b) price intensity theory. However, because this is a preliminary analysis, it is not conclusive. Turning to estimate consistency, Table 1 suggests that the estimates of projects with lower value tend to be more consistent than the estimates of projects with higher value. Number of storeys follows the same trend. The estimates of projects with lower number of storeys are more consistent than the estimates of projects with higher number of storeys. In contrast, GFA shows an opposite trend such that the estimates of projects having lower GFA are most inconsistent when compared with the estimates of projects with larger GFA. For location, the estimates of projects located in the central business district (CBD) are the least consistent when compared with the estimates of projects in the metropolitan and rural areas. The estimates of projects procured with traditional methods are less consistent when compared with the estimates of design and construct projects. Also, residential project estimates are more consistent than industrial project estimates while industrial project estimates are more consistent than the estimates of commercial projects. The analysis also revealed that the estimates of projects constructed with concrete as the principal structural material are the least consistent followed by the estimates of projects

Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011

Building cost estimates

constructed with timber and steel. The estimates of projects with higher price intensity tend to be more inconsistent than the estimates of projects with lower price intensity. Put together, the preliminary analysis suggests that the estimates of small projects are more biased but are more consistent than the estimates of large projects. This means that bias in the estimates of smaller projects is larger and is consistent, whereas bias in the estimates of large projects is smaller but inconsistent.

1263
project value, all the other pairwise differences are small enough that they may be due to chance. There is a significant difference between the mean estimate bias of projects constructed with timber and those constructed with concrete. As before, all the other pairwise differences are small enough that they may be due to chance.

Hypotheses testing
The impact of project factors on estimate bias and estimate consistency The data from the 56 projects were analysed more closely to investigate the trends uncovered by the preliminary analysis (Table 1). Analysis of variance (ANOVA) method was used to compare the mean estimate bias of projects in the different groups of each of the eight factors. The aim was to investigate whether bias in the estimate of the projects varies according to each of the eight factors in Table 1. The results (Table 2) show that there are significant differences between the mean estimate bias for projects in the three categories of project value (F53.27, p50.046). Also, there are differences in the mean estimate bias for projects in the three categories of number of storeys (F511.86, p50.001). Similarly, the mean estimate bias of projects in the three categories of principal structural material are statistically different (F53.96, p50.026). The ANOVA results indicate that estimate bias varies according to project value, number of storeys and type of principal structural material used. After significant differences in the mean estimate were found, the TukeyKramer procedure (Levine et al., 2005) was used to determine which groups are different. The TukeyKramer procedure enabled simultaneous examination of comparisons between all pairs of groups (Levine et al., 2005) for each of the three significant factors (project value, number of storeys and principal structural material). The analysis revealed the following differences:
Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011

Similar to the trend observed in the preliminary analysis (Table 1), the estimates of smaller projects tend to be more biased than the estimates of larger projects. An explanation could lie in the delegation of estimating work. Typically, estimates of smaller projects are undertaken by junior quantity surveyors, while estimates of larger and more complex projects are undertaken by more experienced staff; when junior staff are involved, they are supervised by senior quantity surveyors. Levenes test (Brown et al., 1974) was used to investigate whether the consistency in the estimates varies according to the eight factors. Levenes test enabled analysis of the homogeneity of variance (Conover et al., 1981; Levine et al., 2005). The result (Table 2) shows that the consistency in the estimates did not vary according to any of the eight factors examined (i.e. the variances were found to be homogeneous across the groups of each of the eight project factors). Regression modelling of project factors influencing estimate bias Project factors influencing estimate bias were further investigated using the traditional multiple linear regression technique with the help of the Statistical Package for Social Sciences software (SPSS). The independent/ predictor variables are the eight factors listed in Table 1 and the dependent variable is estimate bias (Y). The factors were entered stepwise. Gross floor area (GFA), project value and price intensity were included in the regression model as continuous variables while other factors were included as categorical variables according to the groupings shown in Table 1. Thus the multiple regression model developed to determine the impact of the factors may be represented as follows: Y ~b0 zb1 GFAzb2 ProjectValuezb3 Price Intensity zb4 CBDzb5 Metropolitanzb6 Rural zb7 Traditionalzb8 D&Czb9 Residential zb10 Industrialzb11 Commercialzb12 Timber zb13 Steelzb14 Concretezb15 1{2Storeys zb16 3{7Storeyszb17 8Storeys and aboveze where: Y5estimate bias;

There is a significant difference between the mean estimate bias of projects having a value of $1$5 000 000 and those having value of $5 000 001 and above. All the other pairwise differences are small enough that they may be due to chance. The mean estimate bias of projects with 12 storeys significantly differs from the mean estimate bias of projects with 3 or more storeys. Similar to

1264
Table 3 Variable Project valuea Number of storeys (37 storeys) Number of storeys (8 storeys and above) Constant Results of multiple regression analysis b Coefficient 20.0318 20.0906 20.0124 0.5811
a

Aibinu and Pasco

Standard error 0.1316 0.0285 0.0478 0.2028

t value 22.41 23.19 20.26 2.86

P value 0.019 0.002 0.797 0.006

R2 (adjusted) 0.2838 F58.27 p50.0001

Notes: Variables are significant at the 5% level (P,0.05).

Continuous data Log transformed in order to normalize data.

b05is the Y interceptit represents the average estimate bias (Y) when all the independent factors in the model are zero; b1 to b17 are the slopes of Y associated with each independent factor when other factors are held constant; e5random error in Y. It is assumed the relationship between Y and the independent variables can be approximated by a linear model which provides best fit estimates of the model parameters by minimizing the error of the model (Draper and Smith, 1981). The regression coefficients b0, b1, b2b17 are unknown parameters. The hypothesis of interest is that: b15b25b35b1750. The predictive performance of multiple linear regression may be judged by the value of adjusted coefficient of determination (R2 adjusted). The results are summarized in Table 3. From the results (Table 3), bias in estimates of projects is influenced by project value and number of storeys. About 29% of the variation in the estimate bias of projects can be attributed to project value and number of storeys. Thus we may conclude that significant variation in early stage estimate bias can be explained by project size. The final regression equation may be written as: Y ~0:5811{0:0318project value{0:0906 3{7storeys{0:01248storeys and above The negative sign of the regression slopes (i.e. 20.0318, 20.0906 and 20.0124) implies that project cost estimate bias tends to decrease as project size increases. This further reinforces the findings from the preliminary analysis (Table 1) and the TukeyKramer procedure which suggest that estimates of smaller projects tend to be more biased than estimates of larger projects. As earlier suggested, this may be explained by experience of estimators. Lowe and Skitmore (2001) stated that estimators prefer the use of individual data and experience. Akintoye and Fitzgerald (2000) found that the three

Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011

main methods of cost estimating are: (1) standard estimating procedure in which construction costs are initially found and allowances for overheads and profit are added; (2) comparisons with past projects based on personal experience; and (3) comparisons with past projects based on documented facts. They noted that these three models are experience based models. Thus there are reasons to believe that the higher level of bias observed in smaller projects might be because such estimates are prepared by junior and less experienced staff. Overall effects of price intensity and type of principal structural material In the multiple regression analysis process, price intensity was entered first to remove any confounding effects. However, price intensity had no effect in the final model. Thus the data did not support the price intensity theory (Gunner and Skitmore, 1999b). The ANOVA analysis shows that estimate bias varies according to the type of principal structural material used (Table 2). However, when put together with the other project factors in the multiple regression model, principal structural material is not a significant predictor of estimate bias. Perhaps the effect was masked by project size (confounding effect) as postulated by Gunner (1997). Has pre-tender building cost estimate accuracy improved over time? A scatter plot was used to test Hypothesis 2 which states that the accuracy of pre-tender cost estimate has not improved over time. The ratio of estimate bias and project value (bias ratio) was determined for each project. Thereafter, the bias ratios of the 56 projects were arranged chronologically according to the time that the estimates were undertaken starting from 1999 to 2007. This yielded a time series dataset. A scatter plot of the time series data was then constructed (see Figure 2). Similarly, consistency ratios were determined for the 56 projects and were

Building cost estimates

1265

Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011

Figure 2 Scatter plot of bias ratio vs. time

arranged according to the time the estimates were undertaken. A scatter plot of the time series data (for consistency ratio) was constructed (Figure 3). If the accuracy of estimates has improved over time, the scatter plots should indicate a downward trend towards zero on the Y axis as we move from 1999 to 2007 on the X axis in Figures 2 and 3. However, the scatter plots suggest that the year of project estimate has no effect on estimate bias and estimate consistency. The trend lines are insignificant. Estimate bias is in the same order of magnitude as it was in 1999, 2000, 2001 and all

through to 2007. The errors are random and inconsistent. Flyvbjerg et al. (2002) found similar results in a study of the differences between estimated cost and actual costs of 258 transportation infrastructure projects. We may conclude that the accuracy of pre-tender building cost estimates has not improved over time. There may be four possible explanations for these results: (1) Estimates of new projects are based on historical cost data from past projects. Thus inaccuracies are transmitted to new estimates over time.

Figure 3 Scatter plot of consistency of estimate vs. time

1266
(2) Firms do not monitor the performance of their estimates in terms of accuracy and so are not aware of any inconsistent error trend. Knowledge of the trend in estimates error and inconsistency in estimates should help firms modify their estimating policy in reaction to the observed inaccuracies and inconsistencies (based on Morrison, 1984 findings). Estimating expertise and skills developed by firms based on experience with past estimates are lost, and so reflect the lack of reduction in estimate bias over time. In Australia, a possible explanation is that there is a shortage and high turnover of quantity surveying and cost engineering skills (reflected by the rating points allocated to the profession by Australias Department of Employment and Workplace Relations (DEWR, 2008) Skilled Occupation Lists for immigration purpose). Skill shortage and turnover may affect the accuracy of estimates when one considers that financial management and cost engineering processes involve assumed knowledge developed over time. New employees, if recruited, will take time to reach their full effectiveness. Other than technical factors such as the skill of the estimating team and their experience, team expertise, estimating techniques, or inadequate data, human-related factors such as the estimators attitude might significantly influence the accuracy of estimates. For instance, estimators are predisposed to increase their estimate (overestimate) when prices are rising pessimism bias; and when prices are falling estimators are predisposed to reduce their estimatesoptimism bias (Mills, 1997). This might be responsible for the lack of trend in the errors (estimate bias) observed and the lack of consistency in the estimates over time. Also the inconsistencies in the estimates might be as a result of wide variance in human judgement suggesting that the human factor is critical when attempting to increase the accuracy of estimates.

Aibinu and Pasco

Relative Effectiveness Index (REI) for each method was determined using the expression (adapted from Kometa et al., 1994): REI~ A B|C

(3)

Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011

(4)

where A5total score; B5highest response options; C5total number of survey responses. The REI was then used to rank the methods. From the results (Table 4), Australian quantity surveyors perceive ensuring sufficient information is available at the time of estimating as the most effective method of improving estimating accuracy (first), followed by increased cost planning and control during the design phase (second) and checking all assumptions with clients and consultants during the estimating period (third). The result is similar to that of Ling and Boo (2001), who found that the most effective methods of improving accuracy according to Singapore quantity surveyors were (a) ensuring design information is sufficient and available for estimate preparation (M3) which ranked first in this study; (b) checking all assumptions when preparing the estimate (M4) which ranked third; and (c) providing a realistic timeframe for estimating activity (M6) which ranked seventh. The similarities indicate an international sentiment regarding methods of improving the accuracy of estimates. Simulation, probability and utility function is considered by respondents as the least effective method of improving the accuracy of estimates (ranked twelfth). Test of agreement among quantity surveyors Fleiss kappa statistical measurement (k) (Fleiss, 1971) was used to ascertain whether quantity surveyors in Australia agree with the ranking of the 12 methods of improving the accuracy of estimates. Fleiss kappa measurement is a variant of Cohens kappa statistical measure of inter-rater reliability. Cohens kappa is suitable where there are only two raters whereas Fleiss kappa can help researchers to assess the reliability of agreement between more than one number of raters on a number of items. If a number of raters assign numerical rating to a number of items then Fleiss kappa statistical measurement will give a measure of how consistent the ratings are. It is a measure of the degree of agreement that can be expected above chance (Fleiss, 1971). Fleiss kappa statistical measurement (k) can be expressed as: k~ P{Pe 1{Pe

Improving the accuracy of estimates To address objective 3 of this study, the respondents were presented with 12 methods that could be used to improve the accuracy of estimates. The methods were identified from the literatureparticularly Ling and Boo (2001). Respondents were asked to rank the effectiveness of each method on a scale of 1 to 5 (where 15least effective and 55most effective). The

where: 1{Pe is the degree of agreement that is

Building cost estimates

Table 4 Effectiveness of mechanisms for improving estimating accuracy Total score Mean One-tailed t-test* (t.3) t-value 157 167 180 172 146 155 134 132 149 169 145 153 4.13 4.18 4.62 4.20 3.65 3.88 3.44 3.38 3.82 4.33 3.82 4.03 6.85 8.23 14.98 7.81 4.11 6.49 2.74 2.07 5.14 9.27 6.57 7.15 p-value 0.0000 0.0000 0.0000 0.0000 0.0001 0.0000 0.0047 0.0230 0.0000 0.0000 0.0000 0.0000 0.826 0.835 0.923 0.839 0.730 0.775 0.687 0.660 0.745 0.845 0.744 0.785 REI

1267

Improvement method IM1 IM2 IM3 IM4 IM5 IM6 IM7 IM8 IM9 IM10 IM11 IM12

Rank

5 4 1 3 10 7 11 12 8 2 9 6

Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011

Notes: REI5Relative Effectiveness Indices of improvement methods. *One-tailed t-test of mean. IM15Ensure proper design documentation. IM25Establish effective communication and co-ordination between members of the project team. IM35Ensure sufficient information is available for estimating. IM45Check all assumptions with clients and consultants. IM55Establish formal feedback for design and estimating activities. IM65Provide a realistic timeframe for estimating activity. IM75Use a more rigorous method of estimating. IM85Incorporate market sentiments and economic conditions into the estimate by way of simulations, probability and utility functions. IM95Incorporate other market sentiments and economic conditions into the estimate. IM105Increase cost planning and control activities during the design stage. IM115Improve methods of selection, adjustments and application of cost data. IM125Update cost database with new cost analyses and provide feedback for improving estimate accuracy.

attainable above chance; P{Pe is the degree of agreement actually achieved above chance. The k statistics can take the values between 0 and 1. According to Landis and Koch (1977) agreement between respondents is Almost Perfect if k50.811.00; Moderate if k50.410.60; Fair if k50.210.40; Slight if k50.00.20 and Poor if k,0.00. In this study, the measurement k is estimated at 0.045 indicating that quantity surveyors only slightly agree with each other on the relative effectiveness of the 12 methods of improving the accuracy of estimates. A possible explanation may be that respondents make use of the different methods to achieve the same result. A one-tail t-test reinforces this conclusion (Table 4). The mean effectiveness for all the methods is statistically above the midpoint of 3 on a Likert scale of 1 to 5 (p,0.05) indicating that quantity surveyors consider all the methods as effective means of improving the accuracy of cost estimates. Reducing estimate inaccuracies in practice Further, the respondents were asked to list internal review mechanisms in place in their company aimed at improving the accuracy of pre-tender building cost estimates. Overall, 33 of 41 survey respondents (80%) listed at least one review mechanism used in their firm. In total, 77 review mechanisms were listed and content analysis was performed on the 77 items. Similar mechanisms were grouped together and named by the researchers. Figure 4 shows the frequency of 13 mechanisms identified from the content analysis.

Overwhelmingly, benchmarking is the most popular method used by firms to improve the accuracy of estimates, followed by internal peer review and communication with the market which both have relatively smaller frequencies when compared with benchmarking. This suggests that estimates are largely based on cost data from past projects rather than on internal peer review and market research. This reinforces the explanation that errors in estimates are transmitted from past projects to new projects. This could be responsible for the lack of improvement in estimate bias over time. The content analysis revealed that the use of computer estimating software (M5) is not frequently mentioned as a method to improve the accuracy of estimates (mentioned by only three out of 41 firms 7%). Also, identification and incorporation of future market trends into estimates (M4) was mentioned only seven times. These results also agree with the ranking of the effectiveness of 12 methods of improving the accuracy of estimates (Table 4) which shows that incorporation of market sentiments into estimates using simulation, probability and utility function (IM8) is the least effective method of improving the accuracy of estimates according to the respondents. Probability estimation or simulation to predict future cost trends or to extrapolate new estimates from past estimates could reduce bias in cost estimates for new projects. However, the results suggest that it is scarcely used. This indicates that there is potentially low uptake of computerized statistical techniques such as cost modelling in the industry.

1268

Aibinu and Pasco

Figure 4 Frequency of mechanisms used by firms for improving the accuracy of estimates
Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011

Key & M1 Benchmarking & M2 Internal Peer review & M3 Communication with the market & M4 Identify and incorporate future market trends into the estimate & M5 Use of computer estimating software & M6 Bulk Checking / Self Checking procedures & M7 Ensuring proper communication / information flow on the project & M8 Use of external price information & M9 Review with final costs on projects & M10 Comparisons with received tenders for future estimates & M11 Elemental review & M12 Internal Quality Assurance procedures & M13 Identification of project specific needs or risks

Conclusion
Data analysis revealed that although bias in pre-tender building cost estimates varies according to project size and principal structural material used, when all factors are controlled and combined in a multiple regression analysis, bias in estimates of projects is significantly influenced by project size. Estimates of smaller projects tend to be more biased than estimates of larger projects. None of the eight project factors studied significantly contributed to the level of consistency observed in project estimates. Thus Hypothesis 1 is partly supported. Firms need to pay greater attention to smaller or less complex projects. When cost estimates of less complex projects are undertaken by less experienced members of staff, they should be reviewed more rigorously by senior and experienced quantity surveyors. The preliminary analysis suggests that bias in the estimates of large projects is smaller but the estimates are inconsistent. As modern projects are becoming more complex, previously used estimating techniques may be inadequate and may not be as effective as they were previously. Quantity surveyors and cost engineers

need to use suitable estimating techniques if cost certainty on projects is to be assured. This study supported Hypothesis 2 which states that the accuracy of pre-tender building cost estimates have not improved over time. Quantity surveyors only slightly agree on the techniques for improving the accuracy of estimates. Thus Hypothesis 3 is not supported. Although firms are using benchmarking of estimates of previous projects to improve the accuracy on new projects (36% of all responses), the effectiveness of such an approach would depend on how often and how accurately cost databases are updated to incorporate market sentiments and economic conditions. It would also depend on firms understanding the size and trend of inaccuracies in their past estimates, the factors influencing the inaccuracies observed and incorporating such knowledge into new estimates. The use of probability estimation and simulation is a way forward in this regard. However, there appears to be a low uptake of this approach according to the respondents. There is need to create awareness of the benefit of computerized statistical techniques such as cost modelling.

Building cost estimates

Firms need to find means of retaining the knowledge and experience gained on estimates of past projects. Rigorous cost analysis and documented feedback from estimates of projects could help firms to pass on the knowledge gained when estimating new projects. Also, firms need to find ways of retaining their staff. Further, quantity surveyors and cost engineers need to be directly involved during the client briefing at the project inception in order that they might adequately understand a clients requirements, rather than depend on relayed information from the project manager or architect. In order to reap and maximize the benefits of cost engineering and quantity surveying skills, clients need to appoint quantity surveyors and cost engineers from project inception. This is one of the few studies on this subject in an Australian context. Its contribution is in the approach used, which involves analysis of real life data and survey data from professionals across Australia. The limitations are acknowledged. The number of projects used for the empirical analysis (56) was not large enough hence small sample groups resulted when projects were divided and analysed according to the eight project factors. This placed restrictions on the ability to detect higher significant effects. However, the results provide a plausible description of the size and the pattern in the accuracy of estimates. The research approach and the step-by-step analysis can serve as a model for others who may wish to conduct similar studies elsewhere on this topic and could facilitate international comparison.

1269
Fleiss, J.L. (1971) Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5), 37882. Flyvbjerg, B., Holm, M.S. and Buhl, S. (2002) Underestimating works in public works projects: error or lie? Journal of the American Planning Association, 68(3), 27995. Gunner, J. (1997) Accuracy of construction pre-bid forecasts, PhD thesis, University of Salford. Gunner, J. and Skitmore, M. (1999a) Comparative analysis of pre-bid forecasting of building prices based on Singapore data. Construction Management and Economics, 17, 63546. Gunner, J. and Skitmore, M. (1999b) Pre-bid building price forecasting accuracy: price intensity theory. Engineering, Construction and Architectural Management, 6(3), 26775. Kometa, S.T., Olomolaiye, P.O. and Harris, F.C. (1994) Attributes of UK construction clients influencing project consultants performance. Construction Management and Economics, 12(5), 43343. Landis, J. and Koch, G. (1977) The measurement of observer agreement for categorical data. Biometrics, 33(1), 15974. Levine, D.M., Stephan, D., Krehbiel, T.C. and Berenson, M.L. (2005) Statistics for Managers, 4th edn, Prentice Hall, Upper Saddle River, NJ. Ling, Y.Y. and Boo, J.H.S. (2001) Improving the accuracy of approximate estimates of building estimates. Building Research & Information, 29(4), 31218. Lowe, D. and Skitmore, R.M. (2001) Human effects in construction contract price forecasting: experience and experimental learning styles. Journal of Construction Engineering and Management, 127(6), 48593. Mills, A. (1997) Analysis of the accuracy of building price forecasts. Australian Institute of Quantity Surveyors Refereed Journal, 1(2), 318. Morrison, N. (1984) The accuracy of quantity surveyors cost estimating. Construction Management and Economics, 2, 5775. Rawlinsons (2006) Australian Construction Handbook 2006, Rawlinsons, Perth. Rawlinsons (2007) Australian Construction Handbook 2007, Rawlinsons, Perth. Serpell, A.F. (2005) Improving conceptual cost estimating performance. AACE International Transactions, EST.13, EST.13.113.6. Skitmore, M. (1991) Early Stage Construction Price Forecasting: a Review of Performance, RICS Occasional Paper, London. Skitmore, M. and Drew, D. (2003) The analysis of pre-tender building price forecasting performance: a case study. Engineering, Construction and Architectural Management, 10(1), 3642. Skitmore, R.M. and Picken, D. (2000) The accuracy of pretender building price forecasts: an analysis of USA data. Australian Institute of Quantity Surveyors Refereed Journal, 4(1), 339. Trost, S.M. and Oberlender, G.D. (2003) Predicting accuracy of early cost estimates using factor analysis and multivariate analysis. Journal of Construction Engineering and Management, 129(2), 198204.

Downloaded By: [HEAL-Link Consortium] At: 12:01 4 January 2011

References
AIQS (2003) Find Member, QS Firms, and Best Practice Charter, available at www.aiqs.com.au (accessed June 2007). Akintoye, A. and Fitzgerald, E. (2000) A survey of current cost estimating practices in the UK. Construction Management and Economics, 18, 16172. Bromilow, F.J., Hinds, M.F. and Moody, N.F. (1988) The Time and Cost Performance of Building Contracts 19761986, Australian Institute of Quantity Surveyors, Canberra. Brown, M.B., Forsythe, A.B. and Alan, B. (1974) Robust tests for equality of variances. Journal of the American Statistical Association, 69, 3647. Conover, W.J., Johnson, M.E. and Johnson, M.M. (1981) A comparative study of tests for homogeneity of variances, with applications to the outer continental shelf bidding data. Technometrics, 23(4), 35161. DEWR (Department of Employment and Workplace Relations) (2008) Skilled Occupation Lists (2008), available at www.immi.gov.au/allforms/pdf/1121i.pdf (accessed 30 May 2008). Draper, N.R. and Smith, H. (1981) Applied Regression Analysis, 2nd edn, Wiley, New York.