2002 AACE International Transactions

CSC.02 Construction Project Performance Metrics

easuring construction project performance is an important piece of the project management/project controls process and must be taken very seriously. The industry has historically used measures of change from the original contracts time and cost as its cardinal performance metrics [8]. While these metrics certainly are wellused and well understood, they suffer from a lack of clarity as to what they actually portray with respect to project performance. The change in a projects time and cost are typically called time growth and cost growth [9, 10]. They are generally represented as either a positive or negative percentage of original contract requirements. While these are indeed useful measures, they do not numerically indicate the reason for the change from original nor do they assign responsibility for that change. For example, a project with a high level of construction cost growth could be suffering from a very poor original design that necessitated the issuance of costly change orders. This is not the fault of the construction contractor. Conventional wisdom would rule that the project is going sour and generally ascribe responsibility to the constructor [3]. On the other hand, the same cost growth could be the function of a costly unforeseen site condition or act of God over which neither party to the contract has control. On the other hand, a negative value of both time and cost growth is generally seen as beneficial, and the project is usually ruled successful [3]. For instance, if an owner decides to remove a feature of work from the original contract, both time and cost growth will become negative because the owner has reduced the scope of work from the original. However, by building less than originally contracted, the project performance metrics will both be negative and the project will be thought to have finished ahead of schedule (negative time growth) and under budget (negative cost growth). The other issue with the use of project performance metrics is their individual applicability for the type of construction that they are measuring. Construction projects can generally be categorized into vertical construction (architectural and other building-type projects) and horizontal construction (highways, railroads, bridges, etc.) [10]. The two types have different contractual bases. Vertical projects tend to have a firm-fixed price contract where the construction contractor must furnish a lump-sum bid for the scope of work portrayed in the plans and specifications and time is measured against a contract completion date. On the other hand, horizontal projects are typically unit price contracts where the contractor will furnish a bid price for each pay item in the

schedule of work and be paid for the actual quantities of work installed. Time is usually measured using charge days (days on which the contractor was able to work) [2, 5]. This difference makes it more difficult for a project manager to measure project performance on horizontal projects than on vertical ones using conventional performance metrics. As contractors bid based on planned quantities and get paid based on actual quantities, the concept of cost growth is no longer valid, as it is entirely possible to have a project that was both properly designed and efficiently built and merely had the actual qualities overrun the planned quantities. Time growth also becomes less discreet. A negative time growth could mean that the owner overestimated the time requirement. This was amply demonstrated in a study of Texas Department of Transportation (TxDOT) projects done in 1995 [4] that found that the average TxDOT project finished nine days early. Thus, project managers or cost engineers must truly understand the dynamics of the metrics that they are using to measure project performance to be able to properly apply them. This is critically important when one looks above the project level and tries to apply these metrics to an ongoing program of many projects [5]. Studies of this nature are often undertaken to evaluate the impact of innovative project delivery techniques such as the introduction of formal partnering or the implementation of DesignBuild (DB) upon a programs performance, with respect to both time and cost. As a result, this paper attempts to statistically evaluate the ability of ten commonly used project performance metrics to actually portray a trend with respect to project performance. The metrics will be defined and then applied to a database of over 1,600 public projects from six different states. Horizontal projects will be separated from the vertical projects to evaluate the value of each metric to measure project performance in these categories. It is hoped that the reader will be able to take this research and be able to more knowledgably apply it to his or her own construction program.

PROJECT PERFORMANCE METRICS There are three types of project performance metrics: relative, static, and dynamic. Relative metrics are expressed as a percentage and as a result are independent of the size of a project. This allows the cost engineer to directly compare the performance of small projects with the performance of large projects. The cost

CSC.02.1

2002 AACE International Transactions and time growth metrics discussed above are typical examples of relative metrics. The second type is static metrics. These metrics are discreet numerical measures that do not change with time. As a result, they are project-size dependent, and the cost engineer can only use them to compare projects that are roughly the same size. Cost per square foot of constructed area and charge days per lane-mile of highway are examples of static metrics. Finally, dynamic metrics are those that vary with time. Dynamic metrics are also project-size dependent. These metrics are generally a function of both cost and time. Some include cost, time, and a function of physical size. As a result of their mathematical complexity, cost engineers must understand the limitations of each and every metric that they choose to use to measure project performance. where: cost growth (percent); original contract cost ($); and final contract cost ($). (equation 2) Award growth is an indicator of the feasibility of awarding a construction project to public companies. It is defined by the difference between original contract cost and the engineers estimate, divided by the engineers estimate. Prior to the advertisement, the engineers estimate needs to be calculated. This estimate is done for the owner and indicates how much money the project will require. The owner then obtains financing in this amount, hoping that the projects actual bid price will be less than the engineers estimate. A positive award growth indicates that the owners financing is insufficient and may have caused a delay while additional funds were obtained. This is especially true for public projects where agencies typically have to return to legislative bodies for increases in project authorizations amounts [6]. Negative award growth indicates that the owner has fenced money against project requirements that were not realized. This often reduces the annual size of an agencys annual construction program by obligating available funding that is not used [6]. Award growth is an excellent measure of how well an owner understands the market in which it must construct its facilities. Award growth = original contract cost - engineers estimate engineers estimate

RELATIVE METRICS The following are three common Relative metrics that were evaluated in this study: time growth, cost growth and award growth. There are others, such as disputes cost percentage, average cost growth due to liquidated damages, and average time growth due to liquidated damage days, which were not evaluated [5]. Time growth is the percentage change in time between the final contract time and the original contract time, expressed as a percentage. Time growth can be positive or negative depending on the outcome of the project. In fact, time growth changes according with the scope of the project. When time growth is positive it means that the project was performed using more time than estimated, and therefore the project was finished late. When time growth is negative the projects time growth was overestimated, and the project was completed before than the original contract time. Time growth = final contract time - original contract time original contract time

where: award growth (percent); original contract cost ($); and engineers estimate ($). (equation 3)

STATIC METRICS Static metrics are size dependent and do not vary with time. While most project managers focus on the elements of construction cost, it is also instructive to include functions of design cost. A poorly developed design will manifest its deficiencies during the construction process. Thus, a life cycle approach to project performance metrics is in order, measuring the projects progress from concept to ribbon cutting. Below are the static metrics that were evaluated in this study. Design unit cost determines the average cost per linear mile for horizontal projects or square foot of constructed area for vertical projects. This value is obtained by dividing the design cost by the size.

where: time growth (percent); final contract time (days); and original contract time (days). (equation 1)

Cost growth is the percentage change in cost between the final contract cost and the original contract cost, expressed as a percentage. Cost growth can be positive or negative, when cost growth is positive there were changes orders increasing the cost of the project during its performance. Otherwise, if cost growth is design cost negative, the original contract cost was possibly overestimated or Design unit cost = size the actual scope of work was reduced. where: final contract cost - original contract cost design unit cost ($/SF or mile); Cost growth = original contract cost design cost ($); and size (SF or mile).

(equation 4)

CSC.02.2

2002 AACE International Transactions Construction unit cost determines the average cost per linear the contractor earns value over the entire period of a construction mile or square-foot of constructed area. It is the measure obtained contract. Effective and efficient construction management is obtained with a high value in construction placement. by dividing the final construction cost by the size. Construction unit cost = final construction cost size where: construction unit cost ($/SF or mile); final construction cost ($); and size (SF or mile). (equation 5) Construction placement = final construction cost final construction time

Where: construction placement ($/day) final construction cost ($) final construction final time (days) (equation 9)

DB placement is the total amount obtained with the sum of Design-build (DB) cost is the total amount obtained with the sum of design cost and final construction cost, which measures design cost and final construction cost divided by the total time period between the start of the design and the completion of the the total amount of the design and construction contract. construction contract. DB cost = design cost + final construction cost DB cost (equation 6) DB placement = design build time (equation 10) DB unit cost can be analyzed for horizontal and vertical projects Construction intensity is an attempt to mathematically comin miles or SF, respectively. It determines the average cost on a project for unit designed and built, which is obtained by dividing bine cost, time, and size into a single metric. It can be used to measure both horizontal and vertical projects. It is the average the DB cost by the size. cost per linear mile or square foot of constructed area per unit DB cost time [8]. Dividing the final construction cost by the size, and DB unit cost = size dividing this final value by the final construction time, obtains intensity. where: DB unit cost ($/SF or mile); Intensity = [final construction cost/size] DB cost ($); and final construction time size (SF or mile). (equation 7) where: intensity ($/SF or mile/day); final construction cost ($); size (SF or mile); and DYNAMIC METRICS final construction time (days). Dynamic metrics are size dependent and vary with time. Dynamic metrics typically combine cost and time and sometimes METHODOLOGY size. They are used to measure the efficiency of project performance [6]. Again, both the design and construction phases of a projEach of the above metrics was calculated for every project in ect can be measured and used to give the cost engineer an idea of how efficiently the work is proceeding. Placement has been used the database. The results of the metrics were then graphed against for years by the US Army Corps of Engineers to measure con- project cost and project time. A metric that shows a strong relationship to either the projects dollar size or its construction peristruction progress against an agency standard [6]. Design placement is the average cost per day of a design con- od is one where the cost engineer can use that relationship to tract. It is obtained by dividing the design contract cost by the establish project financial features such as contingencies and cash flow plans. Linear regression was then used to fit a line to the scatdesign contract time. ter plot, and an R2 value was calculated to indicate goodness of fit. design contract cost Design placement = This value was then used as the measure of a particular metrics design contract time ability to portray project performance. Those metrics that had a where: high relative R2 value were determined to be good measures of design placement ($/day); project performance. The projects in the database were first sepa design contract cost ($); and rated into horizontal and vertical categories to identify those met design contract time (days). rics that best measure performance in each category. Tables 1 and (equation 8) 2 show the results of the metrics tested for the two project categories. Construction placement is the measure obtained by dividing the final construction cost by the final construction time; therefore, construction placement measures the average rate at which CSC.02.3

2002 AACE International Transactions Table 1Horizontal Projects

Design Placement

Construct ion Unit Cost

DB placement 2.42 420 DB placement 2.06

Design Unit Cost

Parameter arameter Average age Skew w R2 (cost) ost) R2 (time) ime) No. of projects of projects

9.32% 4.88 % -1.91% 105,440 2,529,234 6,085,401 2,616,875 44.00% 19.31% 9.82% 192.820 4,698,128 8,386,056 4,892,968 2.52 4.18 -1.25 3.45 3.56 0.4520 0.1497 413 3.27 .9739 0.2153 448 3.56 0.4531 0.1539 411 0.0332 0.0004 0.0231 0.4356 0.0174 0.0001 0.0857 0.1568 543 1,049 401 378

563 751 2.31

21,954 6,112 9,635 21,973 7,568 13,341 2.23 4.38

0.9055 0.4882 0.9273 0.2350 0.1166 0.0074 0.2099 0.0499 437 507 382

Table 2Vertical Projects

Construct ion Unit Cost

Design Placement

Design Unit Cost

Parameter Parameter

Average age Skew w R (cost) ost) R (time) ime)

25.29% 5.21% -3.60% 44.95% 15.57% 25.76% 1.15 5.80% -9.43 0.0127 0.0017 0.0811 0.153 0.0264 0.1426

NA NA NA NA NA

128 25 0.48 0.0175 0.0043

6,043,002 5,497,603 2.19 0.9733 0.0284

134 35 0.46

492 522 2.96

10,866 5,728 8,559 5,267 1.85

NA NA NA NA NA NA

0.5614 0.0359 0.8403 0.4459 0.0331 0.1121 0.0276 0.0215

No. of projects of projects 70 463 381 NA 13 49 6 42 69 48 Note: The data were not available to calculate Design Unit Cost and Intensity for Vertical Projects.

RESULTS By comparing time growth and original construction cost on 543 horizontal projects, it was found that there is not a significant correlation (R2 = 0.033) between these two variables, which means that it is not a good metric to predict the behavior of time in projects performance. This was also the case for cost growth (R2 = 0.0004) and award growth (R2 = 0.0231). Additionally, the large standard deviations and skewness measures indicate that the use of relative metrics is not a function of either project dollar size or construction period. Thus, the idea espoused at the beginning of this paper that the values of these metrics are probably a function of parameters that are not under the control of the construction contractor and/or the owner is supported. The static metrics faired better with R2 values in the 0.4 range. However, this indicates a weak relationship with both dollar size and time. The dynamic metrics had the best correlation with respect to cost. However, this is to be expected, as the largest number in the equation for each metric is the cost number. However, it is interesting to note that the combination of both time and cost improve the statistical correlation for horizontal projects.

The results for vertical projects were even less conclusive. R2 values were only strong for DB cost and construction placement. DB cost can be discounted, as the construction cost is the largest component of the two costs added to arrive at the DB cost. Thus, the R2 value should be nearly 1.00. The R2 value of 0.82 for construction placement, while not statistically significant on a R2 value > 0.95 basis, is still significantly greater than the rest of the metrics. Thus, it can be said that construction dollar size explains about 82% of the variation in construction placement [1].

he comprehensive study of project performance metrics showed that by and large, project performance is largely a function of something other than the projects estimated cost or project execution time period. The value of this study is to clearly demonstrate that the selection of project performance metrics to measure the effectiveness of a specific construction program is not something that can be taken lightly. While it was beyond the scope of this particular study, it can be speculated that if the same analysis were run on projects from a single public owner that the R2 values would increase due to the

CSC.02.4

Intensity

DB Cost

DB Unit Cost

Time Growth

Cost Growth

Award Growth

Intensity

DB Cost

DB Unit Cost

Cost Growth

Time Growth

Award Growth

2002 AACE International Transactions homogeneity of the programs contractual context and the elimination variation due geographic dispersion in the database. Second, it can be concluded that a program manager should select a battery of metrics rather than a single one to measure a large construction program. The evidence given by the measures of dispersion in the various metrics shows that a multiproject program must have several ways to gauge its effectiveness to confidently make informed project management decisions. Finally, the fact that most of the metrics did not correlate with project dollar size or construction period came as a surprise to the authors of this study. The one parameter that could not be measured from this database was a measure of technical complexity of the project. If project performance is not a function of size or time, then the only non-human variable left is technical complexity. An unrelated study done for TxDOT in 1999 found that experienced people who executed a project were the factor that had the strongest correlation with the technical success of the project [7]. While there is nothing in this project performance metrics study to indicate that the quality of the personnel would create a statistical correlation for any given metric, that factor should probably be kept in mind when making business judgments based on project performance metric output. Douglas D. Gransberg, PE CCE Associate Professor of Construction Science The University of Oklahoma Construction Science Division 830 Van Vleet Oval, Room 162 Norman, OK 73019-6141

Monica E. Villarreal Buitrago The University of Oklahoma Construction Science Division 830 Van Vleet Oval, Room 162 Norman, OK 73019-6141

REFERENCES Bluman, Allen G. Elementary Statistics: A Step by Step Approach. New York: Wiley, 1998. 2. Bordelon, J. Alternative Contracting Program-Preliminary Evaluation. Florida Department of Transportation, 1998. 3. Ellicott, M.A. Best-Value Contracting. Proceedings, Area Engineers Conference. TransAtlantic Division, U.S. Army Corps of Engineers, 1994: 9. 4. Grajek, K.M. Partnered Project Performance in the Texas Department of Transportation. Masters thesis. University of Texas, 1995. 5. Gransberg, D.D., W.D .Dillon, H.L. Reynolds, and J. Boyd. Quantitative Analysis of Partnered Project Performance. Journal of Construction Engineering and Management 125, no. 3 (June 1999): 161-166. 6. Gransberg, D.D. The Cost of Inaction: Does Massachusetts Need Public Construction Reform? Boston: Pioneer Institute for Public Policy Research, Center for Restructuring Government, White Paper No. 7. September 1999. 7. Gransberg, D.D., S.P. Senadheera, and I. Karaca. Seal Coat Constructability Program. 1999 AACE International Transactions. Morgantown, WV: AACE International, 1999. 8. Konchar, M., and V. Sanvido. Comparison of U.S. Project Delivery Systems. Journal of Construction Engineering and Management (1998): 435-444. 9. Pocock, J. Comparison of Traditional, Traditional with Partnering and Design-Build Project Performance. Doctoral dissertation. University of Illinois, 1997. 10. Runde, D.F., and Y. Sunayama. Innovative Contractor Selections Methods: Alternatives to the Traditional Low Bid in Massachusetts Public Construction. Policy Analysis Report, John F. Kennedy School of Government, Harvard University. 1999: 46-48. 1.

CSC.02.5