// Does Better Scheduling Drive Execution Success?

Dr. Dan Patterson, PMP

CEO & President, Acumen

Jin Ouk Choi

Research Intern, Acumen

September 2011

+1 512 291 6261 // [email protected]

www.projectacumen.com

Table of Contents
Introduction ................................................................................................................. 3 Hypothesis ................................................................................................................... 3 Approach ..................................................................................................................... 3 Schedule Quality Measurement.................................................................................... 4 Project Execution Measurement ................................................................................... 5 The Projects Used for Assessment ............................................................................... 6 Results & Discussion ................................................................................................... 6 Figure 1 Schedule Quality Index & Finish Compliance Index .................... 6 Does Complex Logic Result in Unnecessary Redundancy? ........................................... 7 Figure 2 Correlation Between Logic Density and Redundancy Index ......... 8 Next Steps .................................................................................................................... 8 Conclusions ................................................................................................................. 9

+1 512 291 6261 // [email protected]

www.projectacumen.com

Introduction
Both the failure and cost/schedule overruns of major CapEx projects receive a high degree of public and stakeholder scrutiny and publicity. However, rarely is the corresponding planning quality and project management maturity given the same level of detailed investigation. Arguably, focus is generally given to the result of failure without also considering the root cause. As such, this white paper is the result of a research project1 that was carried out during the summer of 2011 to investigate the relationship, if any, between project planning quality and project execution success. In other words, this project set out to determine if poor planning results in project cost and schedule overruns and conversely, does sound planning help ensure on time and successful project completion?

Hypothesis
The null hypothesis for this research exercise is that there is no measurable relationship between quality of planning and quality of execution. Instead, the success of execution is driven largely by the contractors ability to execute to a plan irrespective of its realism or achievability. The alternate hypothesis is that there is indeed a positive correlation between sound project scheduling and successful on-time project execution completion, or, the better the plan, the higher the chance of on-time or early completion and the lesser quality of the plan, the higher the chance of a project overrun.

Approach
In order to prove or disprove the hypothesis, a quantitative analysis approach was adopted. So as to establish true quantitative measures for the analysis, the objective of the modeling was to quantify two primary attributes of a project: Quality of the plan Quality of the execution If these two core entities can be successfully quantified, then any correlations between them can be determined easily using standard statistical correlation techniques. To describe this in a more qualitative manner: objectively determine the quality of the plan and compare against the quality of the execution to determine any relationship between the two.

The research project was carried out by Acumen with analysis conducted by Jin Ouk Choi and Dr. Dan Patterson. All rights to the results of this project are the property of Acumen.

+1 512 291 6261 // [email protected]

www.projectacumen.com

Schedule Quality Measurement

Within the discipline of project management, techniques for quantitative tracking of project execution performance are reasonably well established. Accepted approaches such as performance tracking, earned value, earned schedule and progress relative to a baseline are all commonplace today. However, it has only been in recent years that the project management community has recognized the value of applying similar analysis to determine the quality of planning. Through the use of metric analysis (looking at the likes of quality of logic; consistency of detail; appropriate use of activity constraints; use of leads/lags and the resultant impact on float), we now have a means of actually quantifying the quality of a project schedule. Today, there exists several industry standards for schedule metric analysis such as the DCMA 14-Point Assessment as well as thought leaders recommended best practices (see Acumen white paper on metric analysis).2 For this exercise, Acumens extensively used and well-established standard schedule check metric library was used to score the quality of the project plan. These metrics include: Metric
Open-ended Logic

Description
Total number of activities that are missing a predecessor, a successor, or both. This number should not exceed 5% Average number of logic links per activity.

Logic Density

Critical

Number of critical activities

Soft Constraints

Number of activities with soft or one-way constraints

Hard Constraints

Number of activities with hard or two-way constraints

High Float

Number of activities with total float greater than 2 months. This number should not exceed 5% Total number of activities with total finish float less than 0 working days

Negative Float

(ref. D. Patterson, Oct 2009, Project Simplification through Metric Analysis, www.projectacumen.com/resources/whitepapers)

+1 512 291 6261 // [email protected]

www.projectacumen.com

Number of Lags

Total number of activities that have lags in their predecessors. This number should not exceed 5% The amount of redundant or unnecessary logic in a schedule

Redundancy Index

Using this approach, individual metrics are combined into a Schedule Quality Index. This index is a combination of the listed metrics weighted based on their contribution to the structural integrity of a sound project plan. The Schedule Quality Index is based on a 1 to 100% scale with 1% being the lowest quality and 100% being a perfect quality score.

Project Execution Measurement

When selecting a measurement technique for execution performance, there are several choices. Earned value is a sound measure of value created relative to effort/time/cost expended but for the purpose of this research project, the focus is around schedule rather than project cost and so a more schedule-centric method is needed. Traditional performance metric analysis uses simple comparisons such as number of activities that started or finished relative to their corresponding baseline dates. However, a recent white paper3 determined that this type of measurement is not suitable for CPM schedules. The reason being: the measurement is a binary measure that does not take into account how large a slip or acceleration the activity is experiencing. A small one-day slip at the start of the project could cause a domino effect along the critical path causing an erroneous report that all activities on the critical path have slipped. In short: there is no control of granularity using this type of measurement. Instead a measurement known as Baseline Compliance has been developed and is the basis of measurement for this investigation. Baseline Compliance is a measure of how many activities fall within the period that they were expected to fall within. A period is defined by the project based on a standard reporting calendar e.g., weekly or monthly. In the case of weekly reporting, an activity that slips from a planned completion of Wednesday to Friday is not deemed to be late. Fall into the following week however, and its flagged as a slipped activity. This approach is an excellent balance between detailed execution insight and reasonableness of reporting on multiyear projects. Two baseline compliance metrics are available to us: Start and Finish Compliance. Start Compliance is a measure used to determine whether or not
(ref. D. Patterson, June 2011, Baseline Compliance Analysis, www.projectacumen.com/resources/whitepapers)

+1 512 291 6261 // [email protected]

www.projectacumen.com

activities are able to start on time (i.e. are they getting delayed by their predecessors). Finish Compliance is more a reflection of how well activities are being executed and completed. This research project opted for Finish Compliance as the core measurement for execution performance.

The Projects Used for Assessment

Thirty-five projects ranging in value from US$15MM to US$30B were used in the assessment. All projects were classed as major CapEx projects each involving both an owner and (EPC) contractor. The projects are all recent (within the past five years) and nearly all of them are completed. The majority of the schedules were developed by the contractor and then subsequently bought into by the owner going into execution with the schedule as a basis for tracking performance. Projects were modeled in a variety of scheduling tools including Primavera P3, P6 and MS Project. All projects carried an agreed upon baseline schedule which was then used as the basis for calculating Finish Compliance. The results shown in the following section are presented anonymously with no reference to their source.

Results & Discussion

Schedule Quality Index was calculated for each of the projects along with a corresponding Finish Compliance Index. The results were calculated as shown in figure 1.
100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 0%

Finish Compliance Index

R = 0.603

10%

20%

30%

40%

50%

60%

70%

80%

Schedule Quality Index

Figure 1 Correlation Between Schedule Quality Index & Finish Compliance Index

+1 512 291 6261 // [email protected]

www.projectacumen.com

Figure 1 shows a very interesting set of results. Firstly, there is a definite positive relationship between schedule quality index and Finish Compliance Index. As figure 1 shows, as the Schedule Quality Index increases so does the finish compliance index (and vice versa) in essence proving that the better the planning quality, the more activities that finish on time. Secondly, we need to understand how close this relationship is. A trend line was plotted to give an accurate coefficient of determination factor between the two data sets. The R2 factor shows a value of 0.603, meaning there is 60% confidence that a change in the quality of the plan directly drives the quality of execution. Given there are so many other variables affecting project execution (quality of labor, materials, weather, industrial action, owner/contractor relationships etc.), this 0.603 factor is extremely high.

Does Complex Logic Result in Unnecessary Redundancy?

A recently developed schedule metric known as Redundancy Index, measures the amount of redundant or unnecessary logic in a schedule. This metric pinpoints logic links that can be removed with no negative impact on the schedule. These can be removed to simplify the schedule without any changes to CPM results i.e., dates and float remain the same. Removal of these redundancies results in cleaner, more readable schedules that also form a better basis for running risk models. One common question being repeatedly asked has been, If a schedule has a high number of logic links per activity (Logic Density), does this necessarily mean that there is a high degree of redundancy? If the answer is yes, then it makes sense to remove the redundancy. However, if a high logic density does not equate to a high degree of redundancy, then high logic density isnt necessarily a concern. An analysis was conducted on the same project data set comparing Logic Density and Redundancy Index. The results can be seen in figure 2.

+1 512 291 6261 // [email protected]

www.projectacumen.com

35% 30% Redundancy Index 25% 20% 15% 10% 5% 0% 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0 Logic Density R = 0.587

Figure 2 Correlation Between Logic Density and Redundancy Index From figure 2 it can be seen that there is a strong positive relationship between logic density and the degree of schedule logic redundancy. The R2 factor shows a value of 0.6 thus reflecting a very strong correlation between the two. Secondly, from the results, it can be seen that the majority of the Logic Density values fall within a score of 2 and 5. In theory, each activity should have at least one predecessor and one successor, thus driving a minimum score of 2. The results are not only confirming this but also showing that the majority of activities do not surpass an average value of 5. Those activities that do, also then carry a high percentage of redundant logic.

Next Steps
This initial analysis is part of ongoing research within Acumen within the area of project analytics. As we continue to develop more and more analytics, metrics and tools to help with planning and execution, it is equally important that we continue to validate these against real life projects. All of our metrics are continuously being reviewed and calibrated by industry experts and real-life projects. As part of this, we are actively pursuing additional research specifically in the area of logic and float analysis to help form the basis of further insight into CPM scheduling best practices.

+1 512 291 6261 // [email protected]

www.projectacumen.com

Conclusions
From the research conducted, it can be concluded that the alternate hypothesis that there is indeed a positive correlation between sound project scheduling and successful on time project execution completion, is true. While a sound plan cannot be held solely responsible for this driving success factor, it has been shown that it is indeed a highly significant one. Tied very closely to this, overly complex schedules with regards to logic definition also drive a high degree of unnecessary logic redundancy. In summary, projects typically fail due to either unrealistic, poorly thought out plans or weak execution. This research exercise gives a strong indication that by focusing on achieving a sound plan up front; there is a much higher chance of success during execution.

+1 512 291 6261 // [email protected]

www.projectacumen.com