Report to the Independent Board Committee

Report to the Independent Board Committee By Prof. Dr. Flyvbjerg, Bent Prof. Dr. Kao, Tsung-Chung Assisted by Dr. Budzier, Alexander MTR Corporation Limited Electronic copy available at: http://ssrn.com/abstract=2516300 A-2 Contents Page Part I – Executive summary A-6 Part II – Introduction A-19 Part III – Verification of the First IBC Report A-23 Part IV – 2017 delivery A-38 Part V – Project management A-77 Part VI – Budget control A-89 Part VII – Reporting processes and presentation A-96 Part VIII – Recommendations Electronic copy available at: http://ssrn.com/abstract=2516300 A-105 A-3 Glossary Term Audit Committee B3 B4 Board CEO Chairman CIQ CLP CRC CWC D&B Degree-1 D/T DCEO DHy DRM(s) E&M Engineer’s Representatives Entrustment Activities Entrustment Agreement Entrustment Programme Estimated Handover Date ExCo ExCom FD Definition Audit Committee of the Board of MTRCL Basement level 3 Basement level 4 Board of directors of MTRCL Chief Executive Officer of MTRCL Chairman of the Board Customs, Immigration and Quarantine CLP, the Hong Kong power company China Railway Corporation Capital Works Committee Drill and blast Degree-1 structure completion for track-laying and E&M Installation Down track line Deputy CEO of MTRCL Director of Highways Delay recovery measure(s) Electrical and Mechanical The Engineer’s Representative means a person appointed by the Engineer to watch, inspect and supervise the execution of the works and to test and examine any materials or workmanship to be used or provided by contractors. All activities required for the planning, design, construction, testing and commissioning in relation to the Project, including railway works, property development enabling works and miscellaneous works Entrustment Agreement dated 26 January 2010 The programme for the execution of the Entrustment Activities 4 August 2015, being the date set out in the Entrustment Agreement (as may be adjusted in accordance with the terms of that agreement) on which MTRCL estimated that it would formally hand over the completed Project to Government Executive Council of Hong Kong Executive Committee of MTRCL Finance Director of MTRCL A-4 Term First IBC Report GM GM fund Government GRC HSR HyD IAD IBC ICCT Independent Experts International Reference Class July Presentation KCRC KPI M&V Consultant MOR MTRCL OHL OHVD PCG PIMS PjD PjT Procurement Project Projects Division PSC PTC RDO Definition The first report by the IBC on XRL dated July 2014 General Manager (within MTRCL) The budget under control of the MTRCL GM Government of the HKSAR Guangshen Railway Corporation High-speed rail Highways Department of the HKSAR Internal Audit Department Independent Board Committee Initial Contract Control Total The independent experts appointed by the IBC (being Professor Tsung-Chung Kao and Professor Bent Flyvbjerg) A group of comparable, past urban-heavy rail, tunnelling and high-speed rail projects used to benchmark the Project A presentation on the programme status given by the PjT to the CEO, DCEO, and FD on Saturday 13 July 2013 Kowloon-Canton Railway Corporation Key Performance Indicator, an indicator to measure the performance of a contract Monitoring and verification consultant appointed by Government Minimum Operating Requirement. MOR was the term used on occasion by members of the PjT to describe a partial opening scenario for WKT to achieve opening in 2015 (as described in the July Presentation) MTR Corporation Limited Overhead lines Overhead Ventilation Duct Project Control Group Project Integrated Management System Projects Director of MTRCL Project Team for the XRL. The PjT is within the Projects Division Procurement & Contracts Department of MTRCL XRL Project Projects Division of MTRCL Project Supervisory Committee of the XRL Programme to Completion, as of 26 May 2014 Railway Development Office A-5 Term Reference Class Forecasting Reference Class Forecast RCF SCL Service Concession Agreement SIL T&C TBM THB TTMS U/T WKT XRL Definition Forecasting method used to validate the revised cost and schedule forecasts. The method is outlined in Appendix 2. Shatin-Central Link The anticipated arrangement with Government for the operation of the XRL, including running passenger services, after completion of the Project South Island Line Testing and Commissioning Tunnel Boring Machine used in the construction of the XRL Transport and Housing Bureau, Government of the HKSAR Temporary Traffic Management Scheme Uptrack line West Kowloon Terminus Hong Kong section of the Guangzhou-Shenzhen-Hong Kong Express Rail Link A-6 Part I – Executive summary Verification of the First IBC Report Reasons for the delay 1.1 International experience shows that HSR projects are notoriously difficult to build to schedule and cost. It is not unusual for projects of this size and complexity to be subject to delays and cost increases. Building this type of project underground, including a main terminal, in one of the most densely populated urban areas in the world – as is the case for XRL – exacerbates the difficulties. 1.2 The Independent Experts found that the Project has an increased risk profile compared to previous MTRCL projects, because XRL is of a different project type (HSR), requires integration with Mainland rail (cross-boundary issues) and follows a new organisational setup (concession approach). Each of these contributing factors is a “first” for MTRCL, increasing the risk profile of the Project. These are also factors which MTRCL has recognised and addressed. 1.3 Verifying the more specific causes of the XRL delay listed in the First IBC Report, the Independent Experts found ten causes to be particularly important: (1) a fast-tracked front end of the project programme led to late construction start, (2) unforeseen site conditions, (3) late arrival of TBMs from the Mainland, (4) unreliability of TBMs, (5) interface issues, (6) delays at the WKT site, (7) flooding, (8) lower than anticipated production rates, (9) design changes and (10) labour shortages in an overheated construction sector in Hong Kong aggravating the previous items. 1.4 The above issues resulted in a project with a design that was not fully developed before construction started and which, when delayed, instead of being able to catch up with the delay, became increasingly held up as construction progressed. Delay recovery measures 1.5 The Independent Experts observed that the PjT attempted to make up for the delays to XRL through a long list of DRMs. In the view of the Independent Experts, this showed that the PjT was pro-actively addressing the challenges XRL faced. A-7 1.6 However, in hindsight the DRMs were insufficient to finish the Project by 2015. The Independent Experts found that the ineffectiveness of the DRMs later became clear to the PjT and the PjD. However, only the flooding incident at the contract 823A site on 30 March 2014 finally led to the realisation that the delay, which had accumulated to 1.9 years, was irrecoverable. 1.7 It is the view of the Independent Experts that relatively short actual construction schedules for previous MTRCL projects gave the PjT reason to assume a relatively short schedule (5.8 years) at the outset for the Project. However, given site possession and design issues at the start and a very short construction schedule for WKT (3.8 years), delays to the overall Project were going to be very difficult to avoid from when construction began. In the view of the Independent Experts, the negative impact of unforeseen events on the schedule was not so much caused by any flaw in engineering or project management as by a lack of an adequate schedule contingency for critical contracts that would have allowed the PjT to absorb unforeseen events as they occurred. However, the late recognition and reporting that the delay could not be recovered may be seen as a flaw in communicating the project status, in the judgment of the Independent Experts. Project management 1.8 The Project is managed using MTRCL’s Project Integrated Management System (PIMS). The PIMS has been used by MTRCL since 1992. The Independent Experts found that, according to its users, and judged by MTRCL’s past track record, the PIMS has proven effective for previous projects. 1.9 MTRCL internal auditing has been conducted to ensure MTRCL’s conformance to the PIMS. None of the internal audits has found any evidence of significant non-compliance. However, only a self-declaration process by the Project General Manager is in place to assure compliance with the Compliance Manuals. The Independent Experts suggest that two Compliance Manuals should be part of the several levels of auditing in place for the Project going forward. 1.10 Nevertheless, despite the several avenues and layers of systematic reporting and auditing, reporting on the issue of the irrecoverability of delays relied on human judgment within the PjT and, especially, on the PjD and GM. Optimistic assumptions regarding schedule and DRMs led to the realisation that delays were irrecoverable only after A-8 the flooding incident, despite critical voices on the PjT calling attention to the issue earlier in the Project. 1.11 The Independent Experts found that facts about delays were openly reported and the PjT proactively attempted to recover the Project through DRMs. Yet, none of the reporting and audit systems established by the PIMS “raised the flag” of irrecoverable project delay. 1.12 It is the Independent Experts’ conclusion that the interpretation of facts, and the responsibility for making the judgment as to whether the 2015 opening date was to be considered feasible, ultimately rested with a single person, namely the PjD. Moreover, the Independent Experts found that a contributing factor to the PjD not reporting, in a timely fashion, the irrecoverability of the delay was that the current PIMS and Compliance Manuals lack systematic indicators that would prescribe when reporting of delays of a certain likelihood of irrecoverability must take place. Such systematic indicators would have assisted the PjD in making this key reporting decision. 1.13 In sum, the Independent Experts have not seen evidence that the PjT has not followed the systems and procedures of the PIMS and compliance manuals. The Independent Experts have seen evidence, however, that there is scope for improving the systems and procedures as regards systematic reporting on the effect of delays on project programmes. 2017 delivery 1.14 The Independent Experts reviewed the XRL Programme to Completion (PTC) and the Cost to Completion through a top-down and bottom-up verification. The top-down verification was based on a large sample of comparable international projects. The bottom-up verification assessed the risks along the Project’s critical paths. Is the projected completion date at end of 2017 reasonable? 1.15 The Independent Experts conducted a top-down assessment by comparing the PTC to a reference class of comparable international projects. Benchmarking the PTC against the time needed by these projects to finish the final amount of work indicates that the PTC to open XRL for passenger service by 2017 has currently a likelihood A-9 of meeting the schedule of 69%, leaving a 31% risk of delay beyond the end of 2017. 1.16 Benchmarking XRL against previous MTRCL, previous KCRC, current MTRCL and international projects shows that (1) the original and current schedule of the XRL is longer than the average schedule of previous MTRCL and KCRC projects, (2) XRL is of similar schedule length to the other four current MTRCL projects, (3) construction delays in previous MTRCL and KCRC projects were infrequent and minor compared to the international benchmark, (4) delays in current MTRCL projects, except the XRL, are minor compared to the international benchmark and (5) the current delay of the XRL is in line with typical performance of comparable projects in the international benchmark. 1.17 Analysing the PTC bottom-up, the Independent Experts found, that currently the critical path comprises (1) WKT - contract 810A, (2) Tai Kong Po to Shek Kong tunnel - contract 823A, (3) track-laying and E&M installation and (4) testing & commissioning and trial operations. The previously critical cross-boundary tunnel, contract 826, is not at present on the critical path of the PTC but could become critical again due to current TBM issues. 1.18 First, performance of contract 810A is challenged by the lack of progress made in excavating and follow-on concreting of the underground structure of the WKT. Productivity is limited by available spaces for excavating and removing soft grounds and rocks. Currently, the contract is not achieving its planned productivity. In the Independent Experts’ view the contract will be able to increase productivity significantly once diaphragm action at the B3 level of North Top-down Area is achieved. The use of explosives will also de-risk this critical activity. 1.19 Second, contract 823A has only one key risk: the performance of the TBMs. Extensive modifications and improvements have been made to the machines following the drive for the first tunnel. The Independent Experts tested the PTC assumptions and found that, if the improvements are only half as effective as the PjT assumes, the completion of contract 823A would still not impact the 2017 opening date of the XRL. 1.20 Third, track-laying and E&M installation have begun in the finished tunnelling sections. The PjT has taken steps to mitigate the impact of the delay of the civil construction contracts. The Independent Experts A-10 recommend that the E&M team prepares a detailed and wellintegrated contingency plan once the definite concreting plan of WKT is in place. In the worst case, E&M might be able to absorb additional delays of the 810A contract. 1.21 Fourth, testing & commissioning and trial operation have already started. The Independent Experts have confirmed that MTRCL’s Operations Division has a refined roadmap to move forward. However, start of revenue operations in 2017 depends on resolving a list of critical issues with external organisations. The most urgent of these items are: the co-location of CIQ at WKT and signing the Service Concession Agreement. In the Independent Experts’ judgment actions to resolve these issues should not be relaxed due to the change of the opening date to 2017. In particular, the CIQ arrangements need to be resolved so that building designs can be finalised. 1.22 In sum, the top-down assessment shows that the target opening date is more likely to be achieved than not. However, based on the reference class of international projects, we found that 31% of those projects took longer to complete than the current projected delay of XRL. In other words, there is a 31% risk of further delays, judged by the benchmark. Thus, the benchmarking against international projects indicates that the Project is more likely than not to meet the end of 2017 date, and an earlier date may even be achieved if remaining delivery is effective and everything goes according to plan. The Independent Experts find in the bottom-up assessment that the critical contract 810A (WKT) is more uncertain because it has more interdependencies and because as of yet the planned production rates have not been achieved. Similarly, contract 823A has implemented improvements but has yet to prove that the target productivity can be achieved. However, with a well-coordinated contingency plan, including E&M, track installation and operations preparations, the 2017 passenger service is most likely achievable. Is the anticipated budget reasonable? 1.23 The Independent Experts conducted a top-down assessment of the currently forecasted HK$71.5 billion outturn cost by comparing the Project to the International Reference Class. The top-down assessment of the HK$71.5 billion estimate for the XRL showed that further cost increases are likely. According to this analysis, the current cost estimate carries a 67% risk of being exceeded. A-11 1.24 The bottom-up assessment identified that the contingency draw down best reflects the key risks to the cost performance of XRL. The Independent Experts used past patterns of contingency draw down to challenge the Project’s cost estimate. The Independent Experts’ analysis shows that if the current trend of contingency draw down continues the project contingencies will be used up soon and the XRL is likely to exceed its HK$71.5 billion estimate. 1.25 The Independent Experts find that the HK$71.5 billion estimate includes an insufficiently small buffer for unforeseen future events. Based on the top-down and bottom-up assessment the Independent Experts find that the HK$71.5 billion estimate will most likely be exceeded. In the view of the Independent Experts, the main risks to the Project cost are higher future costs of works, liability risks on contested matters and revised assumptions of the current budget forecast. Project management 1.26 The Independent Experts found that a root cause for the delay to XRL was an ambitious schedule of implementation, which did not provide sufficient contingencies for unforeseen events. Initial optimism became apparent when key milestones were missed and assumed rates of production not met. This challenging situation led to a second round of optimism regarding the ability of DRMs to make up for lost time and getting the schedule back on track. 1.27 The Independent Experts found that delays were openly reported and that the PjT proactively attempted to recover the delays through DRMs. However, the Independent Experts observed that after XRL reported an amber traffic light for project progress in June and July 2011, XRL reported 34 consecutive red traffic lights against project progress between August 2011 and May 2014. In the view of the Independent Experts, this indicates that the indicator was not well defined. The Independent Experts recommend using a traffic light indicator reflecting the probability of achieving key dates instead. 1.28 The Independent Experts found that three channels exist in the MTRCL project management process to expose and escalate project management issues. The Independent Experts analysed why none of the channels – the PIMS, the project audits and the project status reporting by the PjT – raised the flag that delays may be becoming irrecoverable. A-12 1.29 First, the PIMS lacks a clear and cogent procedure for escalation of issues of non-performance of projects beyond the PjT. The Independent Experts recommend specific steps to strengthen the reporting in particular to ensure that issues are better escalated in the future. 1.30 Second, project audits currently only focus on assuring compliance to cost control and quality control processes. Programme audits are part of the quality control process. In the view of the Independent Experts, cost and programme are inseparable in project management, the Independent Experts thus recommend to combine the audits of programme and cost under the same body, with reports submitted to the CWC pertaining to its oversight function for capital works. The Independent Experts further found that MTRCL could strengthen the audit channel by building a strong body for independent project assurance that can escalate issues to the Board by enlarging the remit of the internal and project audits. 1.31 Third, the Independent Experts observed that although delays for each contract were factually and openly reported, the interpretation of the consequences of delays and the responsibility for making this judgment rested with a single person, namely the PjD supported by the GM and the PjT. Improved reporting, that would communicate the impact of delays in probability terms instead of on a binary on schedule vs. behind schedule basis, might be more effective in helping those responsible to make the difficult judgment that a project is beyond recoverability. 1.32 Lastly, the Independent Experts recommend that the Board carefully monitors the project management leadership situation through the current transition period of change of PjD and CEO with a view to minimizing or eliminating any risk to the Project from the change in leadership. Budget control 1.33 The Independent Experts reviewed the current budget control arrangements for the XRL. The root cause for the cost overrun is, in the view of the Independent Experts, an initially tight budget. The cost overrun traces back to two factors: (1) delays and (2) market supply pressures in the overheated Hong Kong construction sector. To address the risk of cost overruns in future projects the Independent Experts made three recommendations, namely that MTRCL use: (1) probabilistic cost indicators, (2) Reference Class A-13 Forecasting to establish reasonable budgets and contingencies, as a supplement to current costing practices and (3) a tiered contingency fund for projects under the “concession approach”. 1.34 First, the Independent Experts found that MTRCL project outturn costs are forecast using an event-based analysis. In the view of the Independent Experts, this analysis is prone to optimism, particularly in the assumptions about the future volume and impact of events that may affect a project. The Independent Experts recommend the use of a trending indicator of past contingency draw down to challenge and verify the event-based outturn cost forecast. Such a leading indicator could be reported as the probability of staying within the approved budget. Alternatively, the current event-based analysis should include a pessimistic scenario that assumes that the rate of future claims, variation orders and DRMs is similar to the past rate of contingency draw down. 1.35 Second, the Independent Experts recommend the use of Reference Class Forecasting as a supplement to current costing practices and as an effective method to challenge contingency estimates. The key, and well-proven, advantage is that a Reference Class Forecast is firmly based on empirical data and thus circumvents optimism in assumptions. Had this method been used initially on the Project the Reference Class Forecast would have identified that the original budget had a 60-70% risk of being exceeded. Above, we have used a modified Reference Class Forecasting approach that accounts for the progress made in the Project to assess whether the current budget is reasonable. 1.36 Third, projects under the “concession approach”, i.e. where Government and not MTRCL funds the project, differ from projects under the “ownership approach” by the budget being subject to the political process and greater transparency. Establishing a tiered contingency fund helps to set the right incentives, assure smooth project delivery, while preventing a situation, where project contingencies are used up simply because they are there and their size is common knowledge. A tiered contingency approach needs to be tailored to the specific risks and circumstances of each individual project. 1.37 The Independent Experts find that (1) the recently revised budget is tight due to low contingencies, (2) it will be very challenging for MTRCL to keep within the budget, (3) it is understandable that MTRCL sets a tight budget in order to ensure the best possible outturn cost, but this increases the likelihood of the budget being A-14 insufficient and (4) a contingency for unknown unknowns should be included, some of which could be controlled by Government. Reporting processes and presentation 1.38 The Independent Expert reviewed the reports presented in the project management meetings and the meeting minutes. The Independent Experts found that detailed progress, cost, contractual, environmental and safety information were presented. The Independent Experts also found that these meetings were attended by all levels of XRL staff and stakeholders. However, in the judgment of the Independent Experts, all presentations on the progress of the Project fell short of a tangible analysis to provide participants the opportunity to make a judgment about the effects of the reported delays and the effectiveness of the proposed DRMs. 1.39 The Independent Experts propose to enhance project reporting by introducing simple probability-based indicators. These indicators use programme information already reported in the project dashboards. The Independent Experts recommend to use indicators, such as the Schedule Recovery Index, to communicate by how much the project would have to improve for the remainder of the works to meet the opening date. 1.40 As described above, the Independent Experts observed that XRL reported red lights for a prolonged period in time. To ensure that a red light is meaningful and thus gets the needed attention, the definition of red lights should be tailored to the available buffers and contingencies in a project instead of being consistent across all projects. The Independent Experts further recommend to define clear rules that a project cannot report more than a given number of consecutive red lights, depending on the overall length of the project. Reporting several red lights in a row should trigger a process to review the project. The review should produce an actionable plan for a project turn-around, e.g. how to compress future phases, or rebaselining the project. Also, red lights in a project need to be linked to a clear and transparently communicated plan of actions with clear accountabilities and deadlines to bring project performance back on track. 1.41 Lastly, the Independent Experts considered the level of senior participation in the oversight of MTRCL’s projects. In the view of the Independent Experts, the newly established Capital Works Committee and the Risk Committee have merit in this regard. In the A-15 view of the Independent Experts, the information available in current dashboards and status reports, with the recommended enhancements, would be sufficient for the committees to effectively support and oversee the projects. Recommendations 1.42 The Independent Experts made recommendations regarding (1) enhancing project management to de-risk the critical path contracts, (2) enhancing budget control, (3) enhancing reporting processes and presentation and (4) key reporting milestones. 1.43 First, to de-risk the critical path contracts the Independent Experts recommend that the XRL site team, the contractor and the Government’s engineers work closely and cooperatively as “one team”. Similarly, MTRCL senior management and the Board should establish a close communication between MTRCL, contractors and Government at the most senior level through informal meetings. This should ensure that the best talent and sufficient resources are allocated to the critical contracts, that the best productivity is achieved and that all parties are firmly committed to the 2017 opening date. This should also ensure that the critical outstanding testing and commissioning issues are addressed with urgency. Moreover, an effective and fully-integrated contingency plan is needed for the Project to be ready in case the worst-case scenario develops at WKT. 1.44 Second, the Independent Experts found that the key cost risks for the Project are best reflected in the draw down of contingencies. The Independent Experts recommend introducing a probability-based KPI to track how likely the Project is to stay on budget. Moreover, the HK$71.5 billion estimate should be re-evaluated in the light of the Independent Experts’ assessment. Lastly, going forward the Independent Experts recommend, in particular for “concession approach” projects, that Reference Class Forecasts are used in addition to current cost estimation practices to establish a tiered contingency fund, where different levels of contingency are held by the project, MTRCL and Government. 1.45 Third, the Independent Experts recommend strengthening project reporting through: tailoring the traffic light criteria, using indicators that show the probability of achieving project targets. Moreover, MTRCL should establish an independent channel or strengthen the project audits to escalate issues to top management. This could be A-16 achieved by enlarging the remit of the project audits and providing guidelines for issues to be escalated. The Independent Experts also recommend establishing clear accountability for recovering delays as part of project reporting: red lights in a project need to be linked to a clear plan of actions with clear accountabilities and deadlines for bringing project performance back on track. 1.46 Fourth, the Independent Experts reviewed the PTC and the milestones of the XRL. The Independent Experts, together with the PjT, identified key milestones for the XRL and key KPIs to track progress towards the milestones. The Independent Experts recommend that these milestones and KPIs are closely monitored. Once these milestones are achieved the schedule and cost risks of XRL will change and the remaining risks should be reassessed. Table 1 and Table 2 list the milestones and KPIs. A-17 Table 1. Key reporting milestones to the Board Contract 823A 824 810A 810A T&C 823A 826 810A 826 E&M 824 810A 823A T&C T&C E&M E&M T&C E&M T&C T&C T&C Milestone Re-launching of both north and south U/T TBM Complete D&B excavation Achieve B3 diaphragm action in north top-down area Complete Central Stability System and South Stability System Complete South Stability System statutory inspections Complete south U/T TBM excavation Complete U/T and D/T TBM excavation Erect all roof trusses Degree-1 completion of TBM tunnels for track-laying Complete track-laying for mainline D/T Degree-1 completion of D&B tunnels for track-laying Complete B4 / OHVD and platform track-laying for the first 4 tracks Complete north U/T TBM excavation Energise OHL for pilot section Complete statutory inspections for ventilation buildings Complete track-laying for mainline U/T Energise OHL for dynamic testing of trains in mainline from Nam Cheong to Mai Po, both Up and Down Track Complete statutory inspections WKT Complete WKT track-laying Dynamic testing of Whole Line Dynamic Complete integration T&C with Mainland Commence Trial Running Date Nov 2014 Dec 2014 Mar 2015 Apr 2015 June 2015 Jul 2015 Aug 2015 Dec 2015 Dec 2015 Dec 2015 Feb 2016 Aug 2016 Sep 2016 Sep 2016 Oct 2016 Nov 2016 Jan 2017 Jan 2017 Jan 2017 Feb 2017 May 2017 Aug 2017 A-18 Table 2. KPIs to track key milestones Contract Milestone 824 Complete D&B excavation 810A Achieve B3 diaphragm action in north top-down area 810A Complete Central Stability System and South Stability System 823A Complete south U/T TBM excavation 826 Complete U/T and D/T TBM excavation 810A Erect all roof trusses 826 E&M 824 810A 823A E&M E&M Degree-1 completion of TBM tunnels for track-laying Complete track-laying for mainline D/T Degree-1 completion of D&B tunnels for track-laying Complete B4 / OHVD and platform track-laying for the first 4 tracks Complete north U/T TBM excavation Complete track-laying for mainline U/T WKT track-laying completed KPI Excavation rate Concrete production Steel production Excavation rate Excavation rate Steel production (temporary and permanent) Concrete production Track-laying production Concrete production Concrete production Excavation rate Track-laying production Track-laying production A-19 Part II – Introduction Background 2.1 Following the announcement by the MTRCL on 15 April 2014 of a revised completion date for XRL of 2017 and the resultant public debate, the Board established the IBC at its meeting on 29 April 2014. The IBC consisted of six of the independent non-executive directors on the Board. 2.2 As part of its terms of reference, the IBC was requested by the Board to review the background of and reasons for the revised schedule for the Project. 2.3 The IBC published its first report which was focused on the matters above in July 2014. 2.4 The IBC’s terms of reference also asked the IBC to look forward and advise on how MTRCL can deliver the Project in a transparent and timely manner and in accordance with MTRCL’s obligations under the Entrustment Agreement. 2.5 The IBC appointed two independent experts (the Independent Experts) to assist with its review. This report, authored by the Independent Experts, forms part of the IBC’s second report. The Project 2.6 XRL refers to the 26km long Hong Kong section of the GuangzhouShenzhen-Hong Kong Express Rail Link which will run from West Kowloon in Hong Kong to the boundary of Hong Kong and Shenzhen. XRL will connect with the 16,000km National Highspeed Railway Network in the PRC and is intended to enhance Hong Kong’s role as the southern gateway to the Mainland. 2.7 The Project to construct XRL is large and complex. It involves the construction of a new underground railway system and subterranean passenger terminal in the middle of a densely populated urban area. It requires the engagement and co-ordination of multiple contractors at each of the different phases of its PTC. Finally it involves coordination with Government and other stakeholders. 2.8 XRL is the first railway project to be constructed in Hong Kong under the concession approach. This means that Government pays A-20 for the construction of the railway, bears the construction risk and shares the operational risk of the railway. The ownership stays with Government, while an operator will be invited to undertake the running of the railway. Terms of reference for the Independent Experts 2.9 The IBC asked the Independent Experts to: i. Verify the First IBC Report: a. Review the First IBC Report and assess and verify the findings of the IBC in that report to the extent such findings relate or refer to technical matters or project management procedures and processes regarding the Project; ii. Assess 2017 delivery: a. Consider whether, on the basis of the information currently available, the Expert believes that: (i) the projected completion date of the Project by the end of 2017 is reasonable and (ii) the anticipated budget for the completion of the Project is reasonable; b. Describe key risks that may arise between now and the completion of the Project which may affect completion in 2017 (or completion of the Project on a timely and cost effective basis) and how mitigation for such risks is being implemented (or what additional mitigation measures may be appropriate); c. Identify and recommend key reporting milestones (technical, engineering, E&M, works etc.) in the future project programme for reports to the Board to facilitate monitoring of the progress of the Project toward completion in 2017; iii. Assess and make recommendations regarding project management: a. Taking into account the causes of delay described in the First IBC Report and delay mitigation and DRMs already put in place or developed by the PjT since January 2010, recommend any enhancements the Expert may see as appropriate to the project management of the Project (including reporting to stakeholders) in order to better ensure the completion of the Project in as timely and cost effective manner as possible; A-21 iv. Recommend improvements to budget control: a. Recommend any enhancements to budget control and reporting systems within the project management of the Project; b. Recommend key reporting milestones in the future project programme for reports to the Board to facilitate monitoring of the actual Project budget against the proposed Project budget and anticipated eventual costs outturn; v. Recommend improvements to reporting processes and presentation: a. Recommend any enhancements to the systems within the project management structure for the monitoring and reporting of: (i) progress toward the key reporting milestones and (ii) developments that may affect the seriousness of risks identified already by the PjT or subsequently by the Expert; and b. Recommend any enhancements to the reporting of progress of the Project in order to better ensure transparency and accessibility for all stakeholders or recipients of respective reports. This may include recommendations regarding the presentation and format of such reports as well as their content. 2.10 For the purposes of completing the scope of work described above, the Independent Experts: i. Reviewed the First IBC Report and all underlying documents referred to in that report; ii. Liaised with and/or interviewed members of the PjT; iii. Visited contract area sites; iv. Reviewed documents of MTRCL regarding project management of XRL. These included among others: a. The project procedures and manuals for the Project; b. Internal reports within the PjT (including reports to the PCG); c. Reports produced by the PjT to each of the relevant stakeholders and assurance bodies established within and outside the Project including: the PSC and ExCom; and d. The 2010 Entrustment Agreement. A-22 2.11 The Independent Experts were asked to produce a report setting out the Independent Experts’ findings and recommendations. Acknowledgement 2.12 The Independent Experts wish to acknowledge and are grateful for the co-operation they have received from the IBC and all staff and officers of MTRCL whom they have approached for assistance in the preparation of this report or whom they have asked to give evidence during their review. A-23 Part III – Verification of the First IBC Report Introduction 3.1 The Independent Experts were asked to review the First IBC Report and assess and verify the findings of the IBC in that report to the extent such findings relate or refer to technical matters or project management procedures and processes regarding the Project. 3.2 The Independent Experts' assessment and verification focussed on three areas: (1) the reasons for delay of the Project, (2) the DRMs taken by the PjT to manage the delays and (3) the project management system of XRL. Delay of the Project 3.3 IBC finding 5.1: “The reasons for delay to the Project are numerous. Some apply to the Project as a whole, including labour shortage issues which are affecting all construction projects in Hong Kong, other reasons vary by contract area and may relate, for example, to local geology, different site access issues in the early phases of the Project or unforeseen events occurring. More specifically, for contract 810A (WKT), progress had been and is still affected by unfavourable ground conditions, utility diversion complications, site co-ordination and inadequate work fronts. For contract 826 (cross-boundary tunnelling), progress had been affected by the late arrival of the TBMs from the Mainland. For contract 823A (Yuen Long Tunnel section), progress had been affected by the slow excavation rate of the two TBMs. The flooding of one of the TBM tunnels has made things worse. The key causes of delay in the different contract areas are described in Part IV. These descriptions are based on information provided to the IBC by the PjD and members of the PjT.” 3.4 XRL is the world’s first all-underground HSR line and WKT is the largest excavated underground HSR station in the world. The Project will link Hong Kong to the 16,000 km of HSR in China and create a transport hub in West Kowloon, which will also be home for future commercial development and office buildings. 3.5 International experience shows that HSR, tunnelling and urban heavy rail projects are notoriously difficult to build to schedule and cost. Building this type of project underground, including a main terminal, in one of the most densely populated urban areas in the world – as is the case for XRL – exacerbates the difficulties. A-24 3.6 It is not unusual for projects of the size and complexity of the Project to be subject to delays. In a benchmarking carried out by the Independent Experts, based on a reference class of 59 comparable projects, 7 out of 10 projects (70%) experienced delays; the average delay for these 59 projects was 2.8 years, or +43% measured against the baseline at the time of the decision to build. For previous, completed MTRCL projects (5 total) the frequency of delays was 40% 1 , measured against the planned opening dates in the project agreements. The average delay for all previous MTRCL projects was +3%. Previous KCRC projects experienced delays in 40% of the projects. The previous KCRC projects were delayed by on average +1%. To compare, the delay of the Project is at present 1.9 years, or 32% measured against the opening date in the Entrustment Agreement. 3.7 The Independent Experts verified the immediate causes of the delays to XRL as outlined in the First IBC Report. The Independent Experts found that the Project had an increased risk profile compared to previous MTRCL projects, because XRL is of a different project type (HSR), requires integration with Mainland rail (cross-boundary issues) and follows a new organisational setup (concession approach). Each of these contributing factors is a “first” for MTRCL. 3.8 The Independent Experts observed that, from the start of the Project, MTRCL has taken steps to address these increased risks. MTRCL sought experienced engineers and experts in the construction of HSR projects in order to assist it with the construction of XRL. 3.9 In respect of the risk profile associated with the integration with Mainland rail, the Independent Experts were told that the Project actively monitored issues, such as the progress of the TBMs before handover, and pro-actively solved issues that were arising, such as securing additional spoil disposal site through working with Government and its Chinese counterparts. However, risks remain which will be discussed below. 3.10 In respect of the risk profile associated with the Project following the “concession approach”, the Entrustment Agreement did not require changes to the MTRCL project management structures and processes. MTRCL corporate and project management stated in interviews with 1 These two projects include the Lantau Airport Railway, which was delayed by 6 days to open with the airport and the Quarry Bay Congestion Relief Works, which were delayed by approximately 11 months. A-25 the Independent Experts that they believe that the MTRCL project management system is fit for purpose for projects under the “concession approach”. The Independent Experts observed, however, that the “concession approach” has increased the need for public transparency, scrutiny and communication with Government. 3.11 The Independent Experts further found the following causes of delay to be particularly important by reference to the size of the delays they caused. These causes have been verified by document review and interviews: i. Fast-tracked front end and resulting site possession issues, unfinished detailed design and pending gazettal issues led to late construction start and design changes during construction; ii. Unforeseen site conditions, e.g. unfavourable ground conditions, obstructions in the ground and complicated utility diversion; iii. Late arrival of TBMs from the Mainland; iv. Reliability of TBM; v. Interface issues; vi. Delays at the WKT site; vii. Flooding; viii. Lower than anticipated production rates; ix. x. Design changes; and The above items were aggravated by an overheated construction sector in Hong Kong with labour shortages, including for frontline supervision. Fast-tracked front end and site possession issues 3.12 Compared to the benchmark of international projects, XRL was planned with a shorter than usual front-end for the project programme (as explained further below). In transport projects the acquisition of rights of way prior to construction start is a frequent cause of schedule and budget increases. For Hong Kong rail projects, the front-end process from ExCo policy support to signing the project agreement includes the gazettal of the scheme and the gazettal of amendments to A-26 the scheme. The time between these gazettals reflects the time needed by projects to address objections regarding their environmental and social impact. 3.13 The other four MTRCL projects currently under construction took on average 45 months from ExCo policy support to project agreement, ranging from 38 months (Kwun Tong Line Extension) to 51 months (West Island Line, Shatin Central Link). The international benchmark shows that the average length of the front-end process is 37 months. XRL completed the front-end process in 22 months. This is significantly shorter than the other four projects under construction and the international benchmark. Construction on XRL was planned to start immediately after the Entrustment Agreement was signed in January 2010. The protests and the delayed site possessions demonstrated that, due to the fast tracking, objections of external stakeholders had not been fully addressed. Subsequently, late site possession delayed the start of construction by 225 days (contract 823A) and 130 days (contract 823B), respectively. The Independent Experts understand that this problem was recognised by MTRCL. However, the planned 2015 opening date was not moved. In interviews, the PjT acknowledged that in hindsight MTRCL should have renegotiated the opening date instead of relying on schedule compression. Unforeseen site conditions 3.14 Unforeseen site conditions were encountered in nearly all contracts: i. Contract 802 (Nam Cheong Property Foundation Removal and Re-provisioning) was delayed by deformed H-piles. The piles were expected to be straight but when extraction started the piles were found to be bent like hooks, which required different equipment and a change in extraction method; ii. Contract 803A and 803D were delayed by the unexpected presence of boulders during construction of the diaphragm wall for WKT site 810A; iii. Contract 810B was delayed by unforeseen ground conditions and late diversion of an 11kV cable; iv. Contract 811A was delayed due to artificial obstructions, marine deposits and other ground condition issues; v. Contract 811B was delayed by high rock heads, weak seams A-27 and boulders north of Jordan Road during diaphragm wall construction; vi. Contract 820 was delayed by steel obstructions in the ground. Additional delays were incurred when the TBM encountered voids and suffered ground loss at Hoi Ting Road; vii. Contract 821 was delayed by higher than expected ingress of water and other minor contributing factors; viii. Contract 822 was delayed by poor rock quality and ingress of water; ix. Contract 823A was delayed by high rock levels during construction of the TBM launch shaft. The tunnel drive was further delayed due to shallow soft and mixed ground and air leaks; x. Contract 823B was delayed by high rock levels; xi. Contract 824 was delayed by high water ingress and unexpected amounts of cobbles and boulders during the construction of the Ngau Tam Mei launch shaft; and xii. Contract 825 was delayed due to unforeseen ground conditions and due to a sinkhole forming. Late arrivals of TBMs from the Mainland 3.15 The late arrival of the TBMs from China Mainland delayed contract 826 by 403 days for the D/T tunnel and by 464 days for the U/T tunnel. Initially the TBMs were expected in July 2012 and September 2012; the actual handovers happened on 27 November 2013 and 13 March 2014. Reliability of TBMs 3.16 The Independent Experts understand that contracts 826 and 823A experienced reliability issues with their TBMs: i. Contract 823A was delayed due to breakdowns of the north TBM and was slowed down by low productivity of the TBM. In interviews, the Independent Experts questioned whether the TBM was fit for purpose. The PjT appear independently to have recognised the TBM issues and has continuously carried out improvements to the design and construction of A-28 the TBMs. The TBM issues were further aggravated by the unavailability of engineers from the TBM manufacturer; and ii. Contract 826 was delayed due to a severely damaged TBM, with the damage only discovered upon handover from the China Mainland section. Moreover, repairs and maintenance of the TBM are complicated by the above ground presence of fishponds, which have prevented the PjT from constructing a safe haven for the TBMs. Safe havens are designated zones of the tunnel in which compressed air interventions can be carried out in difficult ground conditions without the risk of air leaks. Safe havens are constructed by grouting a tunnel section from the top. The above ground site restrictions thus slowed down the maintenance and repairs of contract 826. Interface issues 3.17 Interface issues led to a late site possession of Jordan road which delayed contract 811B. Other examples of interface issues are as follows: i. Contract 811B was further delayed due to late utility diversions in 803A; ii. Contract 810B was delayed due to late completion of D-wall in 803D; iii. Contract 811A was delayed due to late site possessions of contract 811B; iv. Contract 810A was delayed at the north top down area due to a combination of delays from 811B; and v. Delays to completion of front end 803A and 803D diaphragm wall contracts at WKT led to delays in contract award and the need to change the scope of the 810B and 810A contracts. Delays at WKT site 3.18 Among the contracts in the Project, 810A (WKT) is the largest, most complicated and most difficult to construct. The contract has been made even more troublesome due to delays on adjacent contracts (810B and 811B), as a consequence of which award of 810A was not possible until 18 October 2011. The contract commenced on 24 October 2011, which left only 3 years and 10 months to complete the Whole of Works for the contract to be ready for passenger service by A-29 August 2015. This was an extremely ambitious, if not impossible, schedule from the outset, in the judgment of the Independent Experts. 3.19 Delay factors at 810A include coupler steel quality issues2, experienced between August and September 2013, and other delays primarily due to interfacing issues, unanticipated movement of the diaphragm wall, temporary works design progress and quality and performance issues with sub-contractors and suppliers. Flooding at 823A 3.20 A “black rain” storm on 30 March 2014 flooded a tunnel at 823A as described in detail in the First IBC Report. The flooding led to a 4-month delay due to severe damage to the electronic components of the TBM in that tunnel. The repairs were finished in July 2014 and the tunnel break-through was achieved in August 2014. Low production rates 3.21 Production rates have fallen short of programme projections in nearly all contracts. This includes quality and performance issues with the Mainland roof fabrication (contract 810A), which were rectified through increased inspection at the fabrication yard. Lower than expected excavation rates have led to delays for contracts 810B, 811B, 822, 823A, 824, 825 and 826. For instance, contract 823A achieved only 30% of the planned excavation rate even before the flooding at the end of March 2014. Design changes 3.22 Early in the Project, design changes have led to delays. Contract 810A has been delayed by the progress of designing temporary works structures. Contract 810B has been delayed by the redesign of the B1 slabs. Contract 811A has been delayed by design changes to the temporary replacement bridges and by design changes to the MKV building. In the view of the Independent Experts, the key reason for the design changes was the fast tracked front-end process of the Project. 2 Couplers are used to couple two steel reinforcement sections before pouring concrete into the structure. A-30 Labour shortages 3.23 Currently MTRCL is developing five projects concurrently, all under construction with the first planned to complete in 2014 (West Island Line). All MTRCL projects are overseen by the Board, the ExCom and the PjD. Moreover, they share central MTRCL functions such as communication, compliance and procurement. In the interviews with the Independent Experts, the PjT and central functions stated that the increased number of concurrent projects has not stretched central resources or slowed down decision making processes. However, MTRCL’s management and the PjT recognise that the large number of concurrent projects might have further aggravated labour shortages experienced in the overheated Hong Kong construction sector and delayed the Project in this manner and prevented DRMs from being fully effective. 3.24 Nearly all contractors experienced labour shortages across all trades and subtrades. On average a 20% labour shortage has been experienced by all five concurrent MTRCL projects, including the West Island Line, South Island Line, Kwun Tong Line Extension, Shatin to Central Link and XRL. In particular, on XRL, contract 822 experienced a shortage of tunnel workers. Contract 824 lacked tunnel supervisors. Contract 823B was delayed by labour shortages at the cut-and-cover segment. 3.25 The above issues resulted in a project, which faced significant challenges to its original programme before construction started and which continuously suffered delays, instead of being able to catch up with the delay, as construction progressed. Delay recovery measures 3.26 IBC Finding 5.3: “The IBC has not identified any systemic flaw in the engineering aspects of the project management process which would suggest that those delays should have been avoided or reasonably could have been handled better.” 3.27 The MTRCL engineering process manages delays by instructing contractors to develop and implement DRMs aimed at meeting the scheduled opening date. 3.28 The Independent Experts observed that the PjT attempted to make up for delays through a long list of DRMs. All contracts, except contracts 802 and 821, developed and implemented DRMs in an attempt to achieve the 2015 opening date. In the case of contracts 823A and B, multiple DRMs A-31 were bundled into Supplementary Agreements with the contractor. The DRMs demonstrate the effort of the PjT to actively try and manage circumstances in order to achieve the 2015 opening date and to mitigate the delays resulting from the specific causes mentioned above. 3.29 PjT was clearly pro-active on this point. In interviews, PjT members made comments such as: “We want to demonstrate that we are not a sitting duck, but that we are compressing the schedule.” Their proactive mindset was also evidenced by the various expert panels and peer reviews for critical construction problems (e.g. tunnelling, roof construction). For example, when the mainland tunnelling section showed delays the MTRCL Project team actively consulted the mainland team to address the performance shortfall, e.g. through working with HK RDO and SZ RDO to enlarge the ground level works area, install additional cranes in the access area and secure an additional spoil disposal site. 3.30 Whilst the proposed DRMs were undertaken in good faith with the clear objective of recovering delays and completing the Project by 2015, in hindsight and taking into account the on-going effect of other delay events, the DRMs implemented were insufficient to finish the Project by 2015. 3.31 The Independent Experts have reviewed the DRMs implemented in each contract area as listed in Table 3. These DRMs cover changes to construction methods, hours, sequencing, layout of work areas and designs. The tunnelling contracts, in particular, made numerous modifications to the TBMs to increase productivity and reliability of the machines. A-32 Table 3. Examples of DRMs (as of April 2014) No. of DRMs used as Contract of 03/14 810A 20 No. of DRMs not used 20 810B 7 1 811A 7 - 811B 20 - 820 4 - 822 2 - 823A 1 - No of future potential DRMs Examples 28 Changes to construction method, changes to temporary structures, changes in struts and D-Wall design, improving works area (e.g. ramps), re-sequencing (e.g. erection before diaphragm action), noise mitigation to extend working hours 29 Change in construction methods (e.g. shear keys), improvement to works access (e.g. ramps), spoil disposal, resequencing (slab construction, Austin Road West Underpass) Extension of work hours, additional D-Wall, changes to methods (bituthene board), design (e.g. base slab area) and layout of construction site Construction method (e.g. top down instead of bottom up for Lin Cheung Rd, pre-splitting, breakwater removal), layout of works area (e.g. temporary traffic flow of Jordan Rd, temporary foot bridge, Lin Cheung Rd flips), resequencing (P-Way access), extended working hours (E&M) and additional resources (D-Wall plant) TBM modifications, re-sequencing of TBM drives and tunnel box construction, adding 3rd shift Enlargement of Ventilation Adit, additional shutters Second TBM, overtime working, resequencing various works, change to tunnel box design (Several DRMs were basis of a Supplemental Agreement; additional DRMs are being implemented now, incl. various improvements to the TBM) A-33 No. of DRMs used as Contract of 03/14 824 3 No. of DRMs not used - 825 3 - 826 5 - No of future potential DRMs Examples Changes to design (height of piles, block walls), relocation of the dismantling area for the TBMs into the tunnel to allow build-out of shaft Use of one additional TBM and conveyor belt system (incl. work gangs, trains, spoil pit), relocation of cross passages to more favourable ground, increase openings to improve access points for track-laying contractors Changes in design (e.g. shortening of section, adding ‘Stage 2B’ for dismantling TBM), re-sequencing (e.g. buffer zone and concurrent cross passage way construction, concurrent construction of invert and walkways), change in construction method (jet grouting instead of bulkhead) 3.32 Ultimately, the net effect of the above issues led to the delay of the opening date of XRL from 2015 to 2017. 3.33 It is the view of the Independent Experts that given the ambitious initial schedule estimate and the site possession issues at the outset, delays were near-certain from the start of construction. In the view of the Independent Experts, the negative impact of unforeseen events on the schedule was not so much caused by any flaw in engineering or project management as by a lack of sufficient schedule contingency that would allow the PjT to absorb unforeseen events as they occurred. However, the late recognition and reporting of the inevitability of the delay may be seen as a flaw in communicating the Project’s progress and status (further below and in Part V). Management of the Project 3.34 IBC Finding 5.4: “The IBC has not seen any evidence to suggest that in their day-to-day work the PjT has not followed the systems and procedures established in accordance with the requirements of the A-34 Entrustment Agreement and vetted by Government and the independent M&V consultant.” 3.35 The Project is managed using MTRCL’s Project Integrated Management System (PIMS). The PIMS has been developed since 1992 and was first used at the Lantau and Airport Railway Project in 1992. Following the KCRC-MTRCL merger in 2006/2007, the PIMS reflects the collection of best practices used at both organisations. The PIMS is continuously updated and improved through changes overseen by a special steering group that meets on a quarterly basis. The aim of the PIMS is to provide a documented management system that meets international standards for quality, safety, environment, risk and asset management. The PIMS is ISO 9001 certified. The PIMS is implemented in MTRCL’s projects through the use of a series of project manuals and practice notes. Projects’ compliance with PIMS is audited through internal audits and self-quality audits. 3.36 The Independent Experts found that according to its users the PIMS has proven effective for past projects under the “Ownership Approach”. Specifically, the PIMS has been successfully used in all nine MTRCL projects since 1994. The PIMS was adapted to the new “Concession Approach” for the Project and two special “Compliance Manuals” based on the entrustment agreements between MTRCL and Government were prepared by MTRCL for its own internal use. These two manuals set out the compliance actions against the requirements under the two entrustment agreements signed with Government on 24 November 2008 and 26 January 2010. 3.37 To assess the suitability of the PIMS for the two concession projects XRL and SCL, MTRCL commissioned independent reviews of the internal controls framework and the PIMS in 2008. These reviews found the PIMS to be fit for purpose and to be complete, robust and comparable to best practice. 3.38 Several levels of auditing, including MTRCL internal auditing and audits by the Government-appointed M&V auditor, have been conducted to ensure MTRCL’s conformance to the PIMS. The M&V audits, as well as the 2013 Internal Quality Audits, Self Quality Audits and Technical Audits, have found no evidence of significant non-compliance. 3.39 The Independent Experts are reasonably satisfied by the audits that the processes of the PIMS have been followed by the PjT. A-35 3.40 For example, the M&V audit reports reviewed by the Independent Experts demonstrated that the auditor challenged the Project on aspects of technical designs as well as programming issues. In April 2013, the M&V list of critical issues included slow progress at contract 826 and warns of a delay of 17-19 months. The M&V list of critical issues from August 2013 estimates a delay of 9.5 months and posed the following question: “…what allowance has MTRCL made in its overall calculations for the critical delays at WKT and Contracts 820 and 826?” The response from MTRCL management was: “The current delays at WKT, 820 and 826 are being monitored closely and their impacts to the Project is being assessed and reviewed.” The M&V audit of 12 September 2013 found that key dates were being missed and that track access had been impacted. The Independent Experts were not presented with evidence that MTRCL’s responses did not satisfy the M&V auditor. 3.41 Similarly, the Audit Committee was briefed by the PjD on the XRL progress. On 7 February 2012 the Committee found that “XRL was slightly behind schedule […] At this point of time, Mr. Chew was still confident [of] a completion by August or September of 2015” and on 14 August, 2013: “In general all five new lines/extensions […] had been progressing in line as planned and project delivery was expected to be on time with some challenges though. Budget management was satisfactory and still under control notwithstanding the volume of claims in place […] there would still be many challenges to overcome […] DRMs were being taken by Management to maintain [the] programme.” 3.42 Nevertheless, despite the several avenues and layers of systematic reporting and auditing, reporting on the issue of the irrecoverability of delays relied on human judgment within the PjT and, especially, on the PjD and GM. 3.43 The Independent Experts found that facts about delays were openly reported and the PjT proactively attempted to recover the Project through DRMs. 3.44 It is the Independent Experts’ conclusion that the interpretation of facts and the responsibility for making the judgment as to whether the 2015 opening date was to be considered feasible ultimately rested with a single person only, namely the PjD. Moreover, the Independent Experts found that a contributing factor to why the PjD did not report the irrecoverability of the delay until April 2014 is that the current PIMS and Compliance Manuals do not include A-36 systematic indicators that would prescribe when reporting of delays of a certain size and likelihood of irrecoverability must take place. Such indicators would have assisted the PjD in making this key reporting decision. 3.45 Instead, the current practice of reporting a delay as being beyond recoverability relies on intuitive judgment by the PjD aided by past experience, supported by the interpretations and experience of other members of the PjT. This judgment call is made all the more difficult due to its binary nature, i.e. a “yes” or a “no” as to whether the schedule may still be recovered. Reversing the Project’s position that the planned opening was no longer feasible and that delays were irrecoverable was particularly challenging given the stakeholders’ expectations. In Parts V-VII below, the Independent Experts recommend improvements to the processes and procedures for reporting on the progress of the Project against its programme. This is done in order to strengthen the existing reporting systems and to ensure that the three available channels (the PjT status reporting, the KPI-based project management system and the internal and external auditors) can “raise the flag” of irrecoverable project delay in future projects. 3.46 In sum, the Independent Experts have not seen evidence that the PjT have not followed the systems and procedures of the PIMS and compliance manuals. The Independent Experts have seen evidence, however, that there is scope for improving these systems and procedures as regards systematic reporting on the effect of delays on project programmes. Conclusion 3.47 The Independent Experts were asked to verify the First IBC Report. The Independent Experts found that the specific causes reported by the IBC indeed resulted in a project, which experienced significant delays to its original programme before construction started and which suffered further delays, instead of being able to catch up with the delay, as construction progressed. 3.48 Moreover, the Independent Experts found that given the ambitious initial schedule estimate and the site possession issues at the outset, delays were near-certain from the start of construction. In the view of the Independent Experts, the negative impact of unforeseen events on the schedule was not so much caused by any flaw in engineering or project management as by a lack of an adequate schedule contingency that would allow the PjT to absorb unforeseen events as they occurred. A-37 The Independent Experts have observed that the PjT was very proactive in pursuing DRMs. However, the late recognition and reporting of the irrecoverability of the delay may be seen as a flaw in communicating the Project’s progress and status. 3.49 The Independent Experts have not seen evidence that the PjT has not followed the systems and procedures of the PIMS and compliance manuals. The Independent Experts have seen evidence, however, that there is scope for improving these systems and procedures as regards systematic reporting on the effect of delays on project programmes. A-38 Part IV – 2017 delivery 4.1 4.2 The Independent Experts were asked to: i. consider whether, on the basis of the information currently available, they believe that: (i) the projected completion date of the Project by the end of 2017 is reasonable and (ii) the anticipated budget for the completion of the Project is reasonable; ii. describe key risks that may arise between now and the completion of the Project which may affect completion in 2017 (or completion of the Project on a timely and cost effective basis) and how mitigation for such risks is being implemented (or what additional mitigation measures may be appropriate); and iii. identify and recommend key reporting milestones (technical, engineering, E&M, works etc.) in the future project programme for reports to the Board to facilitate monitoring of the progress of the Project toward completion in 2017. The Independent Experts reviewed past construction records, reports and the current 2017 PTC. The Independent Experts further conducted site visits and interviews with the PjT and senior managers at MTRCL. Special attention was given to the three critical contracts identified in the First IBC Report (810A, 823A and 826). The Independent Experts investigated the two critical paths identified and the critical activities required to achieve the end of 2017 opening of the passenger service. This section first outlines the 2017 PTC and then verifies this target by two approaches: i. Top-down verification; ii. Bottom-up verification. The top-down verification was based on a benchmarking of the Project against a large sample of comparable international projects. The bottom-up verification assessed the risks of the Project along the critical paths. A-39 Is the projected completion date at end of 2017 reasonable? 4.3 XRL was originally planned to open in August 2015. On 15 April 2014 MTRCL announced a revised opening date for XRL at the end of 2017. Subsequently, a PTC was developed detailing all the works necessary to take the Project from April 2014 to its new opening date at the end of 2017. 4.4 The PTC is driven by two distinct critical paths, as depicted in Figure 1: i. Degree-1 3 completion of contract 823A north U/T tunnel by September 2016 for track-laying and E&M installation; and ii. Completion at B4 level of contract 810A for track-laying by August 2016. This will allow commencement of dynamic testing of the train and E&M system for three months from February 2017. This will be followed by an additional three months of test runs and three months of trial runs, totalling nine months. XRL is planned to open to passenger service by the end of 2017. 3 Degree-1 completion refers to the handover of station or building areas from civil works to E&M works and tunnels from civil works to trackwork. At Degree-1 completion all civil works (concreting and wet trades like plastering, painting and floor screeding) should effectively be ready for commencement of E&M works or trackwork as relevant. Degree-1 completion will not include work items such as floor tiling, walls and ceilings which are part of the civil scope of works carried out subsequently. A-40 Figure 1. XRL Programme to Completion (PTC) 4.5 Development of the PTC incorporates the lessons learned over the past 3.5 years and assumes more achievable rates of productivity than previously. The PTC is founded on: i. Better understanding of the geological conditions; ii. Adjusted assumptions about labour supply, based on current market conditions; iii. Adjusted assumptions about rates of productivity, based on previous achieved rates; and iv. Improvement of equipment design (e.g. TBMs) and its supportive accessories, based on their past performance. Plan for contract 810A according to PTC 4.6 Contract 810A was challenged from the beginning due to the short time for initial preparation. While most of the XRL contracts commenced in 2010, the designs for contract 810A were still in development and were being coordinated with E&M works, and addendums to the tender were still being issued. Subsequently, when work on contract 810A commenced after October 2011, the contract was already more than one year behind most XRL contracts and the A-41 original schedule. This gave contract 810A only three years and ten months for completing the excavation of four underground floors, building the large underground station structure, laying 10 tracks, constructing 21 platforms, erecting an intricate roof structure, installing building services and E&M facilities, testing and commissioning the railway, and finally providing cross-boundary passenger services. 4.7 With all these sequential and highly interdependent construction works having to be implemented in a very limited time span and confined space, the Independent Experts question whether the initially expected target date of passenger service in August 2015 had a high likelihood of being achieved from day one of contract 810A. No matter how good the PjT’s intentions were and how hard they tried to recover delays, they had an extremely challenging task from the outset for this contract, in the view of the Independent Experts. 4.8 The PTC of contract 810A incorporates the lessons learnt from the scheduling of the original programme including regarding achievable production rates and the labour situation. The critical path at the WKT site is programmed with the following construction sequence and key dates: i. Achieve B3 Diaphragm Action in North Top Down Area – March 2015; ii. Erect all roof trusses– December 2015; iii. Complete B4, OHVD and platforms for track-laying, first four tracks – August 2016; iv. Power on for North CLP transformers – August 2016; v. Track-laying completed in WKT – January 2017; vi. OHL energised for WKT platform tracks – March 2017; vii. Complete construction of station entrance – April 2017; and viii. Complete access road to station and open Lin Cheung Road underpass – April 2017. A-42 Plan for contract 823A according to PTC 4.9 The two key factors that caused the delay at 823A were: (1) late land resumption at Choi Yuan Tsuen, which caused half a year of delay and (2) performance shortfall with TBMs caused by design deficiencies and the response to maintenance needs. Consequently, actual production rates of the TBMs were below planned rates. For the 880 metres (440 rings) of the north D/T tunnel, the actual construction period was 653 days. The actual average rate of tunnelling was 4.8 rings per week (0.7 rings per day). The actual achieved tunnelling rate was very low compared to other XRL tunnelling contracts and also compared to the originally planned rate. 4.10 The contractor on 823A is now carrying out an extensive modification of the north TBM and plans to increase the production rate by 114%. The north U/T tunnelling is planned to take 300.2 days according to the PTC. 4.11 The key dates for the PTC for contract 823A are: i. Initial U/T drive – 31 March 2015; ii. U/T tunnelling completed – 7 April 2016; iii. Degree-1 completion of the U/T tunnel – 17 September 2016; and iv. Track-laying completed – December 2016. Plan for contract 826 according to PTC 4.12 In the original 2015 programme, contract 826 was on the critical path mainly due to: i. A 14.5 month delay in the arrival of the TBM from Mainland; and ii. Slow progress of tunnelling works in a marble zone due to the area being classified as environmentally sensitive, prohibiting surface geotechnical drillings along the tunnel alignment. 4.13 According to the PTC, contract 826 is planned to achieve Degree-1 completion for both the U/T and D/T tunnels by September 2015. This would leave ample buffer for the E&M installation and tracklaying in 2016 and dynamic testing starting February 2017. The A-43 Independent Experts therefore consider that, for the time being, contract 826 can be dropped from the critical path list for the PTC. Only in the case of unexpected delays on contract 826 would this contract re-enter the list. 4.14 Although sufficient buffers are available to contract 826 for now, the 826 team should continuously exercise its diligence to prevent this contract from becoming critical again, and the PjT should diligently monitor this. The current unexpected cutter head damage of the D/T TMB should be recovered as soon as possible. Top-down assessment 4.15 In interviews with MTRCL staff the Independent Experts asked what the P-value4 of the end of 2017 schedule is, i.e. the likelihood that the schedule will be met. The Independent Experts were told that the Pvalue is 90% (P90), i.e. the schedule will be met with 90% certainty, leaving a 10% risk of delay beyond the end of 2017. 4.16 In order to reach its own assessment of the probability of the Project achieving the 2017 completion date, the Independent Experts benchmarked the remaining part of the Project, as of July 2014, against: i. A reference class of similar international projects; ii. Previous, completed MTRCL and KCRC projects; and iii. Current, on-going MTRCL projects. 4 The certainty of a forecast is denoted by the P-level. Forecasts generate several estimates. Conventional forecasting techniques produce these estimates through simulations. Reference Class Forecasting, as recommended below, produces these estimates based on the variation of actual, observed data in a reference class of similar projects to that being forecasted. For example, P50 is the middle estimate. P50 is defined to be an estimate where 50% of the observations in the reference class fall below this estimate and 50% fall above this estimate. For a P40 estimate 40% of the observations in the reference class fall below and 60% fall above this estimate. A P90 estimate means that 90% of the observations do not exceed the estimate. Thus the P-level of the forecasts gives an indication of the certainty and the risk of the estimate, a P40 estimate expects that the actual value is less than the forecast with 40% probability and that with 60% probability the forecast will be exceeded. Similarly a P90 estimate has a 90% certainty, with 90% probability the forecast will not be exceeded and with a 10% probability the forecast will be exceeded. It is important to note that this does not mean that the P90 estimate is the estimate that is most likely to happen, P90 is an estimate that will most likely not be exceeded. The P50 estimate is the one most likely to happen. A-44 Benchmarking against international projects 4.17 The benchmarking against international projects is shown in Table 4. The benchmark consists of 85 high-speed rail, tunnel and urban rail projects that can be considered comparable to the Project in terms of technology used and planning regimes and for which data on schedule were available 5 . Statistical tests showed no significant differences between the three types of projects regarding schedule; the data for the three types can and should therefore be pooled, as done in the table row with the total. Table 4. Construction schedule length, international benchmark High-speed rail Tunnels Urban heavy rail Total XRL Construction schedule length, years (average) 8.5 6.0 7.2 7.0 5.8 (7.9) 4.18 The benchmarking shows that the average construction schedule length in the benchmark was 7.0 years (median 6.1 years). This compares with an initial estimated schedule length for the Project of 5.8 years, extended to 7.9 years in July 2014. 4.19 The benchmark in terms of schedule risk is shown in Table 5. Measured against the agreed opening date set out at the decision to build, the benchmark shows that 7 out of 10 (71%) of the projects were delayed, with an average schedule overrun of 43%. Again, the different project types are not statistically significantly different. Against the benchmark, the Project at present has a schedule overrun of 36%. It should be kept in mind, however, that all the projects in the benchmark are 100% completed, whereas XRL is currently 60% complete as of July 2014. 5 The Independent Experts considered urban light rail, urban heavy rail, high-speed rail and tunneling projects. Statistical tests showed that urban heavy rail, high-speed rail and tunneling projects are comparable in terms of schedule length, schedule overrun and cost overrun. Further tests were conducted to ensure that projects from different geographical regions and different time periods are comparable. The test results concluded that urban light rail projects ought to be excluded from the reference class and that all other projects are comparable in relation to schedule duration and schedule and cost overruns. A-45 Table 5. Schedule risk, international benchmark High-speed rail Tunnels Urban heavy rail Total XRL Frequency of Size of schedule schedule overrun overrun (average) 76% +44% 57% +31% 75% +50% 71% +43% nr +36% 4.20 When the projects in the benchmark were 60% complete, they needed on average 2.8 years (median 2.4 years) to reach full completion. The XRL plans to complete the remaining 40% of the project in 3.2 years. If the XRL delivery team performs no worse or no better than delivery teams on the projects in the benchmark, then there is a 31% risk the XRL team will not meet the end of 2017 completion date for XRL. 4.21 The Independent Experts believe there are two risk factors associated with the Project which would suggest that the 31% probability of not meeting the 2017 schedule might be conservative (not exaggerated), i.e. the real risk may be higher. These are that: (1) Given that the XRL is substantially more complex (all underground, including a large underground terminal, all built in a dense urban area) than the majority of projects in the benchmark; and (2) XRL has experienced an unexpected change in top management (CEO, PjD) midway through construction (discussed further below). Due to delays, the disbursement profile for the Project (see Figure 2) is different from the typical disbursement profile for similar projects. This fact also would suggest that a conservative interpretation of the 31% value is appropriate in the view of the Independent Experts. A-46 Figure 2. Typical disbursement profile versus XRL profile 4.22 As shown in Figure 2, the typical disbursement profile follows an “Scurve” with a slow start and ending to disbursements during start-up and winding down of construction, respectively, and accelerated disbursements in between. In contrast, disbursements for XRL are following a "hockey-stick curve" with a slower-than-typical rate of disbursements in the beginning and linear disbursements after that, assumed to continue at full rate with no slowing down until the end of construction. As a result, at the current level of progress for XRL, which is 62.3%, in a typical project approximately 25% of the risk would still be outstanding, whereas for the XRL the figure is 55%, making the project more risky than average, in the view of the Independent Experts, other things being equal. Figure 3. XRL progress, planned and actual A-47 4.23 The reason for the particular risk profile of XRL is shown in Figure 3. Initially the Project assumed typical S-curve progress. Progress of the PTC is now planned to be more linear, with a lower rate of progress than planned. Again, this indicates that the Independent Experts’ top-down risk assessment is conservative because both the schedule (Figure 3) and the cost (Figure 2) profile show higher remaining risks for the Project compared to the typical benchmark project. Benchmarking against completed MTRCL and KCRC projects 4.24 The benchmarking against previous, completed MTRCL and KCRC projects is shown in Table 6. The benchmark consists of five previous MTRCL projects completed after 1998 that cost more than HK$0.5 billion in 2012 prices. Those projects were Lantau Airport Railway, Tseung Kwan O Extension, Quarry Bay Congestion Relief Works, Disneyland Resort Line and Tseung Kwan O South Station. The benchmark also includes five KCRC projects for which data were available. Those projects were: Kowloon Southern Link, West Rail Project, Ma On Shan Rail, Sheung Shui to Lok Ma Chau Spur Line and the Tsim Sha Tsui Extension. Since 1998, MTRCL also completed two smaller projects, which were not included in the benchmark – namely, the Asia World-Expo Station and the Sky Plaza Platform. 4.25 The benchmark in Table 6 shows that previous MTRCL and KCRC projects were shorter than XRL. On average MTRCL projects took 3.6 years from the start of construction to opening and KCRC projects took 4.1 years. Table 6. Construction schedule length, MTRCL and KCRC projects Construction schedule length, years (average) XRL 5.8 (7.9) Previous MTRCL projects* 3.6 Previous KCRC projects 4.1 * Completed 1998-2014, larger than HK$0.5 billion A-48 4.26 The benchmark in Table 7 compares the actual opening date with the estimated handover date set out in the Entrustment Agreement. On average previous MTRCL projects were delayed by +3%, and 40% of these projects were delayed. The two projects that were delayed were the Lantau Airport Railway, which was delayed by only 1 week, and the Quarry Bay Congestion Relief, which was delayed by approximately 11 months. On average KCRC projects were delayed by +1%, and 40% of these projects were delayed. In comparison, the XRL schedule overrun is currently 36%. Table 7. Construction schedule risk, previous MTRCL and KCRC projects Frequency of Size of schedule schedule overrun overrun (average) XRL nr +36% Previous MTRCL projects* 40% +3% Previous KCRC projects 40% +1% * Completed 1998-2014, larger than HK$0.5 billion 4.27 Table 8 compares the front-end process and the construction schedule between the different reference classes. The results show that XRL only took 21 months from ExCo policy support to signing the entrustment agreements. This is significantly shorter than the previous and current Hong Kong rail projects as well as the international benchmark. It is important to note that in 3 of the 5 previous MTRCL projects and 4 of the 5 KCRC projects construction actually started before the project agreement. If construction did not start before project agreement, construction commenced immediately at agreement6. It is also important to note that, due to changes in the front-end procedures, only current MTRCL projects are strictly comparable to the XRL. As discussed above, during the front-end process, initial designs and impact analyses of projects are disclosed to the public and objections to the projects are addressed. The delays due to late site possession, unfinished detailed design and design changes during the construction of the Project, as discussed in Part III, are a result of the fast-tracked front-end. 6 The longest time between project agreement and commencement of construction was at Tseung Kwan O Extension, which took 5 days. A-49 Table 8. Average length of the front-end (in months) Time from ExCo policy support to Project Projects Agreement Construction time XRL 21 70 (93) Current MTRCL 46 67 Previous MTRCL* 35 45 Previous KRCRC 44 50 Benchmark 37 84 * Completed 1998-2014, larger than HK$0.5 billion Benchmarking against current MTRCL projects 4.28 The benchmarking against current, on-going MTRCL projects is shown in Table 9. The benchmark consists of five projects: West Island Line, South Island Line (East), Kwun Ton Line Extension, Shatin Central Link East West Line and Shatin Central Link North South Line. For all projects, except the Shatin Central Link North South Line, construction has commenced. The projects are at present between 93% (West Island Line) and 17% (Shatin Central Link) completed. The benchmark shows that the projects are on average planned to take 6.1 years in construction. Table 9. Construction schedule length, current MTRCL projects Construction schedule length, years (average) 5.8 (7.9) Projects XRL Current MTRCL, 6.1* excl. XRL * Planned construction schedule length as of August 2014 4.29 Table 10 shows that 100% of the MTRCL projects currently under construction are delayed. The average delay of these projects is +11% with further delays being expected. A-50 Table 10. Schedule risk*, current MTRCL projects Frequency of Size of schedule Projects schedule overrun overrun (average) XRL nr +36% Current MTRCL, excl. XRL 100% +11% * Planned construction schedule length as of August 2014 4.30 Table 11 summarises the results of the schedule overrun benchmarks. Table 11. Summary of schedule overrun benchmarks Frequency of Size of schedule schedule overrun overrun (average) nr +36% XRL Current MTRCL projects*, 100% excl. XRL Previous MTRCL projects** 40% Previous KCRC projects 40% International benchmark 71% * Planned schedule as of August 2014 ** Completed 1998-2014, larger than HK$0.5 billion +11% +3% +1% +43% 4.31 Given the results of the above benchmarkings, the Independent Experts find: • The original and current schedules for XRL was/is longer than the average schedule for previous MTRCL and KCRC projects; • The schedules for current MTRCL projects are of similar length to that of XRL; • Construction delays in MTRCL and KCRC projects have been infrequent and were only minor delays compared to the international benchmark; • Delays in current MTRCL projects other than XRL are minor in comparison to XRL and the international benchmark; and • The current delay of XRL is in line with typical schedule performance of comparable projects in the international benchmark. A-51 4.32 In sum, the Independent Experts observe that: First, construction schedules for MTRCL projects have historically been short but have become longer on recent projects. While MTRCL might originally have expected to be able to repeat with XRL its past performance in achieving short construction schedules, records show that the XRL schedule is today closer to a typical international project of this type. Second, high risks remain for completing the Project. Comparing the remaining part of XRL to typical projects, the Independent Experts find that the end of 2017 schedule is more likely to be a P70 than a P90, i.e. the current XRL schedule is likely to be met with 69% certainty instead of 90% certainty, leaving a 31% risk of delay beyond 2017. In other words, the Independent Experts find that the Project is more likely to be finished by the end of 2017 deadline than not, and it may even be completed ahead of schedule if everything goes according to plan and the remaining part of the Project is managed effectively. Bottom-up assessment 4.33 The bottom–up assessment will examine the two critical-path activities from end to end to evaluate the risks in achieving them. The following sequence is applied: i. Contract 810A; ii. Contract 823A; iii. Track-laying and E&M installation; and iv. Testing and commissioning and trial operation. Contract 810A 4.34 Among the two critical paths identified in the 2014 PTC, contract 810A for WKT is the more complicated in terms of engineering constraints, the sequential nature of the works, the size and scope, the interfaces with adjacent contractors, the TTMS arrangements and the necessary utility and traffic diversions. It incorporates all the functions and complexities of an underground tunnel railroad, a mega station, a new underpass and a commercial development. A-52 4.35 In the PTC, an average monthly concrete pouring rate of 20,000 m³ between now and April 2016 is assumed. This production rate forms the basis of the PTC at 810A. This concreting rate takes into account the soil/rock excavation still to be carried out under the limited space of the top-down area. In July 2014, the actual concreting rate achieved was 17,000 m³. This rate was achieved with limited work space in the top-down area. Excavation progress is expected to open up additional work fronts, i.e. make additional floor areas available for concreting, thus increasing the pour rate. The planned average rate of about 20,000 m³/month was based on the 810A contractor’s programme submissions. 4.36 The assumption that additional work fronts will be opened up, and that this will create more excavated areas available for concreting, is reasonable. However, the Independent Experts consider the one-off concreting rate of July 2014 to be insufficient proof that the high concreting rate assumed in the PTC can be achieved right away. As of August 2014 this rate has not been achieved. Delay in achieving the planned production rate of 20,000 m³/month will impact two important follow-on key dates: (1) achieve B3 diaphragm action in north top-down area by March 2015 and (2) complete B4, OHVD and platforms for track-laying the first four tracks by August 2016. 4.37 Although the current production rate is still much below the PTC planned production rate, the site team is currently working with the contractor to open up new work fronts in order to achieve the production rates required. The Independent Experts agree that once the “bottle neck” of working in a confined work space can be resolved, the project might reach the planned production rate. However, this depends on how soon the B3 diaphragm action can be achieved7. Once the B3 diaphragm action is achieved, the concrete production can be increased significantly. 4.38 In the Independent Experts’ assessment, the achievement of the B3 diaphragm action is at risk of being delayed beyond the original planned March 2015, due to the current shortfalls in productivity. However, once diaphragm action is achieved, the site team should 7 Diaphragm action of the floor provides lateral support to the walls. It enables the excavation below the floor in the top-down construction method. Achieving the diaphragm action at level B3 also significantly reduces the risk of ground settlement at the WKT site. A-53 have a reasonable chance to catch up with the production rates and will not significantly delay the completion of B4 for the track-laying by August 2016. 4.39 Achieving the planned concreting rate for the WKT station construction critically depends not only on (1) the ability to excavate and remove material in the confined site of the top down area, but especially also on (2) the rate of excavation and removal of the bedrock, once this has been exposed. To ensure that the planned production rate is achieved, the 810A site team has applied and obtained a permit to use explosives for the rock excavation once the blasting programme is finalised and fully approved after site testing, the effect of rock blasting will be evident. 4.40 However, in the Independent Experts’ view, a well prepared contingency plan should be prepared in case the target dates are not achieved. If the above efforts still fall short of assuring the handover of B4 area to the track-laying team by August 2016, a wellcoordinated contingency plan with the subsequent track-laying, E&M installation and testing and commissioning activities is needed to accommodate the potential delay. With this contingency plan the 2017 passenger service is still achievable. The Independent Experts have discussed with the XRL E&M team the need for this contingency plan. The E&M team is responsible for all activities after the handover of B4 floor and platform sites. Currently, the contingency plan is being considered while the E&M team awaits the finalisation of the detailed plan for the construction of contract 810A. 4.41 Although not on the critical paths, there are two additional significant risks at the 810A site that should be mentioned. They are: (1) construction of the terminal roof and (2) repeated relocations of the Lin Cheung Road. These do not directly affect the track-laying works, E&M installations or the testing and commissioning of the high-speed trains. However, both carry risks that may impact a smooth commercial service beginning in 2017. The Independent Experts recommend that the Board closely monitors the construction progress of the terminal roof and supports the site team by working to have Lin Cheung Road closed. A-54 The roof 4.42 The key risks in respect of the roof construction are: (1) production of the steel for the roof is located in Thailand and China, where quality issues have been found in inspections, and (2) the PjT’s lack of previous experience in erecting similarly complex roof structures. To mitigate these risks, the site team has increased the coordination and quality inspections at the manufacturing sites and has prepared and practised detailed computer simulations of the installation steps for the roof. 4.43 In the view of the Independent Experts, the PjT follows good project management practice by having developed two contingency plans for the worst case scenario, in which not all of the roof can be constructed according to the planned schedule. Lin Cheung Road 4.44 For the construction of the western side of the station and the Lin Cheung Road Underpass the current Lin Cheung Road needs to be relocated several times in and out of the construction area. These relocations pose significant potential risks not only to the construction schedule but also to the safety of the road users all of which have to be managed closely. The site team is currently pursuing closure of the Lin Cheung Road. 4.45 In sum, the Independent Experts find that, although contract 810A still has not been able to achieve the planned production rate, with new work fronts opening up and the development of a contingency plan to coordinate the B4 handover and track-laying, the 810A PTC plan is achievable. Moreover, blasting to accelerate the rock excavation and closure of Lin Cheung Road will significantly de-risk the 810A PTC. 4.46 Although the PTC provides a feasible plan for the construction of 810A, the programme remains very tight and critical, with little slack. The site team of MTRC and the contractor must work closely and cooperatively as “one team” to face the challenges and remain effective and efficient. The Independent Experts recommend that MTRCL senior management invite senior management of the Contractor to jointly commit the best resources to the site and to reconfirm their commitment to achieving the key milestones. A-55 Contract 823A 4.47 The risks at the 823A site are more straightforward and focused, compared to contract 810A. There is one key issue: performance of the north TBM. 4.48 Figure 4 shows the originally planned drilling rate of the north D/T TBM at 2.7 rings/day. However, due to a lower actual production rate, breakdowns of the machine and flooding, the tunnelling rate actually achieved was only 0.7 rings/day. The current planned tunnelling rate, assumed in the 2014 PTC, is 1.5 rings/day. This requires an increase in the efficiency of the north TBM of 114%. Figure 4. Planned and actual tunnelling rate, north TBM (rings per day) 4.49 According to the site team, this planned 114% efficiency increase in the tunnelling rate will be achieved through modifications and improvements to the north TBM. The planned improvements and modification and their effects on reducing the construction period of the north U/T tunnel are estimated by the PjT as follows: i. TBM modification to reduce breakdowns: 103.5 days; ii. Prevention of the flooding risk: 93 days; iii. Tunnelling efficiency increase due to new cutter head: 51.9 days; iv. Tunnelling efficiency increase due to new screw conveyer: 39.4 days; v. Tunnelling efficiency increase due to new man lock arrangement: 46.8 days; A-56 vi. Tunnelling efficiency increase due to new heat exchanger and slurry circulation system: 10.2 days; and vii. Tunnelling efficiency increase arrangement: eight days. due to better logistic 4.50 The overall shortening of the construction period for the north U/T tunnelling, compared with the north D/T drive, is therefore estimated at 352.8 days, with a total tunnelling period of 300.2 days. 4.51 Based on the above estimate, the Independent Experts summarised the improvement of the average daily tunnelling rate planned for the TBM as described in Figure 5. Figure 5. Expected improvement of the tunnelling rate, north TBM (rings per day) 4.52 The Independent Experts consider the measures taken to minimize the risk of another flooding or similar incident to be comprehensive. However, the Independent Experts stress-tested the other assumptions behind the PTC by assuming only half the improvements to tunnelling compared with the assumptions of the PjT. In this case, the overall improvement in speed would reduce from 114% to 64%, as shown in Figure 6. Under this more conservative assumption, the average tunnelling rate would be 1.15 rings/ day and the total tunnelling period would be 382 days. A-57 Figure 6. Conservative assumption for improved tunnelling rate, north TBM (rings per day) 4.53 When the original PTC was planned, it was still unknown to the PjT how soon the flooded TBM could be recovered. Start of drilling for the U/T TBM was assumed to be 31 March 2015. However, the team recovered the TBM sooner than expected and the estimated start of drilling moved forward to November 2014. This change has created a buffer of around four to five months before the follow-on tracklaying activity is scheduled. Under the conservative assumption of an 82 day longer period for drilling, contract 823A would still have two months of buffer. 4.54 Therefore, achieving the key dates for contract 823A mostly relies on the performance of the TBMs. To secure high performance, the site team has agreed with the contractor and the TBM manufacturer to station a testing and commissioning team of four to five engineers and technicians, one tunnel supervisor and one tunnel E&M engineer on site to support U/T tunnelling until completion of the drives. 4.55 The Independent Experts conclude that the key handover date of 823A to track-laying and E&M installation is achievable, but is still exposed to substantial risk, i.e. reliability of the TBM. To minimise this risk and achieve the projected completion date, the Independent Experts recommend that the Chairman and/or CEO of MTRCL contact the Chairman and/or CEO of the Contractor and the TBM Manufacturer to ensure that there is commitment and pressure from the most senior management to keep the TBMs running with minimal delay until completion. A-58 Track-laying and E&M installation 4.56 Following the Degree-1 completion of the civil works, the track and overhead lines can be installed. These works are carried out by the XRL E&M team, which is also in charge of station E&M, building services, depot equipment and rolling stock testing and commissioning. The scope of the team's work include: i. Track work installation; ii. Building services; iii. Rolling stock testing and commissioning; iv. Power system installation; v. Signalling system installation; vi. Communication control; vii. Ticketing system installation; viii. Security monitoring system installation; ix. Depot equipment procurement and installation; and x. Maintenance equipment procurement and installation. 4.57 Due to the delay of the civil construction works in tunnelling and at the WKT, the E&M team has already undertaken several remedial measures to proactively prevent the civil delay from further negatively impacting schedule and cost. These remedial measures include: i. Procuring special track-laying plant to allow track-laying without using work trains. Conventionally E&M activities are carried out sequentially by running multiple specialised work trains along the full length of the track. For the Project, special purpose-built lifters have been constructed to allow track installation in completed tunnel sections so that the team does not have to wait until full segments of the tunnel are handed over; ii. Changing track construction sequence to suit the availability of sites; A-59 iii. Rescheduling the manufacturing schedule for bulky equipment in order to store equipment underground now, which avoids future bottle necks at the access points to the tunnels; iv. Locating storage area for equipment that has already been shipped; v. Advancing cable installation manually instead of awaiting for track-laying and work train installation; vi. Shifting key plant rooms away from delayed WKT areas to reduce the impact of the delayed hand over; vii. Reducing the onsite construction period through off-site prefabrication and testing of some subsystems; viii. Using direct labour instead of subcontractors for 810B, where installation is available in a small and piecemeal manner to save contracting time and cost; ix. Coordinating with civil contractors to identify key rooms for E&M installation to assure that critical areas of WKT are handed over as they become available; x. Verifying software compatibility at site office to reduce the site interface testing period; xi. Conducting sub-systems integration off-site to reduce the onsite construction time; xii. Conducting the high-speed dynamic testing of first train on the Hu-Kun Line in China to reduce the dynamic testing period; and xiii. Applying for labour under the Supplementary Labour Scheme (SLS) now to have additional resources for later stages that will require a large amount of E&M installation works to be carried out concurrently at WKT. 4.58 The Independent Experts have examined these measures and judge them to be a reasonable and effective way to deal with possible further delay of the WKT B4 handover for track and E&M installation. The Independent Experts recommend that once the definite concreting plan for WKT is in place, the E&M team works A-60 with the 810A team to put in place a contingency plan for the worstcase scenario of a delay to the handover for track-laying in order to ensure that this would have no negative impact on meeting the opening date for passenger service at the end of 2017. Testing and commissioning and trial operation 4.59 After completion of track-laying and E&M installation, the follow-on major activities are testing and commissioning and trial operations, which are planned to start in February 2016 and to finish to be ready for the end of 2017 opening to passenger services. 4.60 The MTRCL Projects Division will lead testing and commissioning, supported by MTRCL’s Operations Division. During trial operations, and operations themselves, the MTRCL Operations Division takes the lead in planning, execution, coordination and monitoring. 4.61 In the original XRL programme, with the estimated handover date in August 2015, many critical issues remained unresolved, which jeopardised the handover date. While many issues are still unresolved, the new PTC affords additional time to resolve the issues. 4.62 MTRCL’s Operations Division has developed a comprehensive plan to guide its operation preparation works consisting of 15 major tasks and 751 subtasks and activities. 4.63 Among the necessary operation preparation works, the Operations Division has identified 60 key action items and among them ten items are classified as “very challenging”. Successful on-time completion of all of the ten very challenging action items requires coordination, commitment and agreement from outside organisations by, for example, Government, GRC and CRC. Resolving these items requires continuous discussion, coordination and conciliation to find solutions in time so that follow-on activities are not delayed. 4.64 Currently, the most urgent action item to be resolved is the decision on co-location of the CIQ. The CIQ arrangement will have repercussion effects on many follow-up activities such as the interior design of WKT, the operation procedures, the profitability of XRL, the time table and train arrangement. Most urgently, the uncertainty around the CIQ arrangements prevents the building designs from being frozen, which might cause further delays and additional costs in the future. A-61 4.65 The Service Concession Agreement with Government also ought to be sought as early as possible. This will enable MTRCL to commission the operator-operator communication between Hong Kong and the Mainland. This is necessary for MTRCL to: i. Develop a co-operating agreement with GRC; ii. Develop joint operation procedures with GRC for both normal operation and incident handling at the interface tunnel section; iii. Agree on maintenance demarcation on system-wide equipment and at the interface tunnel section with GRC; iv. Tender pre-ops maintenance and services contracts, as well as the commercial contracts; and v. Agree with CRC/GRC on operation and maintenance staff training and qualification for both Mainland and HK staff. 4.66 The other “very challenging” items which require joint efforts of Government, GRC, CRC and MTRCL are to decide the Safety Management System, Certification Principles, Revenue Split, etc. 4.67 The Independent Experts’ investigation has confirmed that the Operations Division has a refined roadmap to move forward. Given the time available the roadmap is achievable. However, the resolution of all these “very challenging” items is not fully within the control of MTRCL. Resolving them requires continuous dialogue, persistent coordination and reasonable conciliation. Therefore actions should not be relaxed due to the delay of the opening date to 2017. 4.68 While all “very challenging” items require extensive coordination and discussion, the rest of the testing and commissioning and trial operation activities requires skills which MTRCL already has and are similar to testing and commissioning and trial operations in the existing MTRCL systems. Past MTRCL experience may need to be modified in some cases to suit the high-speed rail operation (such as the ticketing system and cross-boundary operations), but in the Independent Experts’ view this should prove fairly straightforward to the PjT given its previous experience. 4.69 For past MTRCL projects, the testing and commissioning and trial operations were scheduled to last six months, comprising two months of dynamic testing, two months of integration testing and two A-62 months of trial operations. For the Project, a nine-month period (three + three + three months) has been allocated. The Independent Experts consider this assumption, with a third extra time, to be prudent. 4.70 If the “very challenging” action items requiring coordination, commitment and conciliation with the relevant outside organisations can be resolved in time the planned milestones for testing and commissioning and trial operations are likely to be achieved. Conclusions, bottom-up assessment 4.71 The PTC is driven by two distinct critical paths: i. Completion of B4 level at 810A for track-laying by October 2016. This will allow commencement of dynamic testing for the train and E&M system from February 2017 for three months, which will be followed by another three months of test running and three months of trial runs (three + three + three months), before opening to passenger service by end of 2017; and ii. Degree-1 completion of 823A north U/T tunnel by September 2016 for track-laying and E&M installation. 4.72 The Independent Experts have used the bottom-up assessment to examine these two critical paths from end to end. The Independent Experts conclude that: i. Critical path # 1 (WKT 810A) is more uncertain because it has more interdependencies and because as of yet the production rates assumed in the PTC have not been achieved. However, with a well-coordinated contingency plan, including E&M, track installation and operations preparation, the 2017 passenger service at WKT should be achievable, in the view of the Independent Experts, despite substantial remaining schedule risk; and ii. Critical path # 2 (823A) is also achievable for 2017 passenger service, provided remaining schedule risks are mitigated effectively. A-63 4.73 The Independent Experts recommend, specifically for contract 810A, that: i. Corporate-level attention and support are given to the key tasks of the contract, aimed at achieving the key dates. The best talent and resources of MTRCL and the WKT contractors should be allocated to the contract and all parties should be firmly committed to the opening date; and ii. An effective contingency plan is prepared and ready in case the worst-case scenario develops at WKT, including the involvement of all relevant experience and knowledge of the civil construction, E&M and the Operations teams. Is the anticipated budget reasonable? 4.74 This section first outlines the currently forecasted cost to complete XRL and then verifies this target by two approaches: i. Top-down verification; and ii. Bottom-up verification. Current forecasted cost to complete XRL 4.75 Procurement provided the Independent Experts with the details of the most recent forecast of the XRL outturn cost of HK$71.5 billion, which is based on the May 2014 cost report. 4.76 The current outturn cost forecast for XRL is the result of a collaborative effort by Procurement and the PjT. The forecast outturn cost comprises the current committed cost and the future cost risk. Following MTRCL practice the future cost risk falls into two categories: i. Allocated risks, which reflect the estimated cost risk of completing the Project in accordance with the PTC for the inauguration of passenger services in 2017; and ii. Contingency, which reflects an allowance for future unknown events. A-64 4.77 According to MTRCL the current outturn cost forecast for XRL is based on the following major assumptions: i. Date for inaugural passenger operations moved from 2015 to 2017; ii. All costs are assessed based on the applicable contract terms and valuation principles; iii. No cost allowances made for recovery of damages where delay may be the fault of the contractor; iv. No changes in project scope; v. For variable price contracts (mainly tunnels), future escalation risk based on Government projections of inflation; and vi. No allowance for special bonus or commercial incentives for contractors. 4.78 According to Procurement, the outturn cost forecast for XRL will be continuously developed and updated on a six monthly cycle, in accordance with MTRCL's established practice and procedures. Top-down assessment of budget realism 4.79 In interview with members of Procurement, the Independent Experts asked what the P-value of the revised cost estimate of HK$71.5 billion is, i.e. the likelihood that the budget will be met. Procurement’s response was that its analysis took into account all the identified risks and their assessed quantum values on which it ran Monte Carlo simulations. The values taken were based on a 90% confidence level. Hence Procurement’s confidence in meeting the currently projected outturn cost of HK$71.5 billion is 90% (P90), based on the assumptions and related identified risks, leaving a 10% risk of exceeding that estimate. If these assumptions and the related risks change this will affect the forecast. 4.80 In order to assess the revised cost estimate, the Independent Experts benchmarked the remaining part of the Project, as of July 2014, against: i. A reference class of similar international projects; ii. Previous, completed MTRCL and KCRC projects; and iii. Current, on-going MTRCL projects. A-65 Benchmarking against international projects 4.81 The benchmarking against international projects is shown in Table 12. The benchmark consists of 112 high-speed rail, tunnel and urban rail projects that can be considered comparable to the Project and for which data on cost were available. Statistical tests showed no significant differences between the three types of projects regarding cost overruns; the data for the three types can and should therefore be pooled, as done in the table row with the total. Table 12. Cost performance, international benchmark High-speed rail Tunnels Urban heavy rail Total XRL Frequency of Size of cost cost overrun overrun (average) 93% +41% 66% +38% 73% +45% 76% +42% nr +10% 4.82 The benchmarking shows that the average cost overrun in the benchmark was +42% (median +30%). This is 32 percentage points higher than the cost overrun of +10% for the Project. It should be kept in mind, however, that the projects in the benchmark are all completed, whereas XRL is currently only 60% complete. 4.83 At 60% complete, the estimated total outturn cost for XRL is HK$71.5 billion. If the Project performs no worse or no better than the projects in the benchmark, then there is a 67% risk the XRL team will not meet the estimate of HK$71.5 billion for the Project. 4.84 The Independent Experts believe there are two risk factors associated with the Project which would suggest that the 67% probability of exceeding the currently projected cost outturn is conservative, i.e. the real risk may be higher. There are that: (1) XRL is substantially more complex (all underground, including a large underground terminal, all built in a dense urban area) than the majority of projects in the benchmark; and (2) XRL has experienced a change in top management midway through construction (discussed further below). Due to delays the disbursement profile for the Project is different from the typical disbursement profile for similar projects. This fact also would suggest that a conservative interpretation of the 67% value is appropriate, in the view of the Independent Experts. A-66 Benchmarking against previous MTRCL and KCRC projects 4.85 The benchmarking against previous, completed MTRCL and KCRC projects is shown in Table 13. The benchmark consists of five previous MTRCL projects completed after 1998 that cost more than HK$0.5 billion in 2012 prices. Those projects were Lantau Airport Railway, Tseung Kwan O Extension, Quarry Bay Congestion Relief Works, Disneyland Resort Line and Tseung Kwan O South Station. The benchmark also includes five KCRC projects for which data were available. Those projects were: Kowloon Southern Link, West Rail Project, Ma On Shan Rail, Sheung Shui to Lok Ma Chau Spur Line and Tsim Sha Tsui Extension. 4.86 The benchmark shows that previous MTRCL projects achieved lower or no cost overruns compared to XRL. 20% of the MTRCL projects exceeded their budget, that is only one out of the five projects: the Lantau Airport Railway. On average previous MTRCL projects had a cost overrun of -17%, that is, on average previous projects stayed within budget. All KCRC projects stayed within budget. On average they underspent the budget by 11%. Table 13. Cost overrun benchmark of previous MTRCL projects Frequency of Size of cost cost overrun overrun (average) XRL nr +10% Previous MTRCL projects* 20% -17% Previous KCRC projects 0% -11% * Completed 1998-2014, larger than HK$0.5 billion Benchmarking against current MTRCL projects 4.87 The benchmarking against current, on-going MTRCL projects is shown in Table 14. The benchmark consists of four projects: West Island Line, South Island Line (East), Kwun Tong Line Extension, Shatin Central Link. The benchmark shows that two of the projects (West Island Line and South Island Line) have announced that they will exceed their budgets. The average cost overrun for the four projects is +5%. According to MTRCL staff, the supply pressures in the overheated HK construction sector are the key reason why MTRCL projects now are more similar to the international benchmark. A-67 Table 14. Cost performance, current MTRCL projects Frequency of Size of cost cost overrun overrun (average) nr +10% 50% +5% XRL Current MTRCL projects*, excl. XRL * against latest approved baseline 4.88 Research has shown that the key error in bottom-up forecasts is the underestimation of variance between forecasts (cf. Appendix 2). Table 15 compares the differences between the P50 and P90 levels of certainty of the estimate. The results show that the variation between the estimated outturn cost of the Project is in line with the variation that would be expected given the current MTRCL projects. The variation in the current outturn cost forecast of the Project is narrower than estimates based on the behaviour of previous MTRCL and KCRC projects. Most notably though, the top-down forecast, i.e. using the International Reference Class, shows a ten-times larger variation than the bottom-up forecast predicts. Above it was documented that, at present, schedule overrun for the Project is more than three times higher than for other current MTRCL projects, and cost overrun twice as high. Moreover, it was concluded that regarding schedule and schedule overrun, the Project is more similar to projects in the international benchmark than to other Hong Kong projects. In conclusion, this means that most likely the P80 and P90 cost risks and budget estimates for the Project, as provided by Procurement, are underestimated at present, in the view of the Independent Experts. Table 15. Range between P50 and P90 of forecasted outturn cost XRL* Previous MTRCL Previous MTRCL & KCRC Current MTRCL Previous and current MTRCL & KCRC Reference class * based on August Cost Report Difference (% of the base estimate) 3% 10% 6% 3% 5% 31% A-68 4.89 Given the results of the above benchmarkings the Independent Experts find: • The 10% cost overrun of the Project is lower than the typical cost overrun in the international benchmark; • Cost overruns in previous MTRCL projects have been rare and were minor compared to the international benchmark. Previous KCRC projects had no cost overruns; and • Cost overruns are currently being experienced in half of the on-going MTRCL projects. However, the size of the overruns is smaller than that of the international benchmark and similar to XRL. 4.90 In sum, the Independent Experts found that while MTRCL has a reputation for completing past projects on budget, cost overruns are now happening in its projects, although they are smaller than in comparable international projects. While MTRCL might well be able to repeat past performance of low or no cost overruns, XRL today appears to face cost risks that are more typical of projects in the international benchmark than of previous MTRCL projects. Even with the recent 10% increase in the XRL budget, the Independent Experts find that high cost risks remain for the completion of the Project. 4.91 Comparing the remaining period of the Project to performance in typical international projects, the Independent Experts found that the HK$71.5 billion cost estimate would be more likely a P33-budget, i.e. the certainty of the budget being met would be 33%, leaving a 67% risk of further cost increases, if the Project continues to perform like international projects. Based on the results of this benchmark, it is the view of the Independent Experts that XRL would have to achieve levels of performance for the remainder of the project that are more in line with previous MTRCL projects, and less with international ones, if the XRL is to stay within the current budget. Bottom-up assessment of budget realism 4.92 To verify the top-down assessment of the HK$71.5 billion estimate the Independent Experts analysed the XRL contingency draw down. According to the contingency reported by Procurement, the Project has nearly depleted its contingency reserve. 4.93 The MTRCL contingency management process works as follows: a project budget is established based on the estimated cost plus an A-69 allowance for contingency. This budget is sub-divided into budget headers, which align with the anticipated works contract packages. When a contract is awarded, the budget for that contract is reconciled against the awarded contract sum, this is referred to as the ICCT. Where the contract sum exceeds the budget allowance the difference is drawn from contingency; where the contract sum is less than the budget the difference is returned to contingency. Where it is necessary to increase the ICCT for a given contract, a change form is submitted to the PCG with justifications as to the reasons for the increase, together with details of the quantum. The ICCT plus any approved changes which are funded from contingency are referred to as the current control total. Where there is a potential cost increase that is not yet seen as a commitment, this is referred to as a potential change. The current control total plus potential changes, which are funded from contingency, are referred to as the estimated final cost. 4.94 In order to challenge and verify the bottom-up cost estimate of the Project the Independent Experts extrapolated the historic trend of contingency draw down into the future. The forecast shows that, if the Project follows past patterns of contingency draw down, the remaining contingency will be used up by mid-2015. The forecast also shows that by the opening date in December 2017 the cost would exceed the current estimate of HK$71.5 billion. This forecast is based on the assumption that the past contingency draw down is systemic for the Project, i.e. that unforeseen changes to the ICCT remain constant. Thus, the forecast is likely to be conservative, in the view of the Independent Experts, given that several civils contracts are nearing completion and given that high cost risks still remain for the unfinished tunnel excavation work and the WKT construction, as described above. 4.95 As described above, applying the top-down benchmark approach suggests that at a 50% level of certainty (P50) the current projected outturn cost will be exceeded. The Independent Experts found that the top-down and bottom-up verifications of the Project outturn cost are very similar and support each other well. Key cost risks 4.96 The review by the Independent Experts identified three project risks related to cost: (1) the current contingency position, (2) uncertainty of the funding situation and (3) risk of escalation. A-70 4.97 As described above, the reported contingency provisions comprise allocated and unallocated risks. The HK$71.5 billion estimate (based on the May 2014 cost report) includes: i. Currently committed cost, i.e. the sum of the newly adjusted contract values (the known knowns); plus ii. Allocated risks, which reflect the estimated cost risk of completing the XRL in accordance with the programme for the inauguration of passenger services in 2017 (the known unknowns); plus iii. Actual contingency (the unknown unknowns). 4.98 In the view of the Independent Experts, there is a significant risk that the contingency will be insufficient for the remaining three years until opening for passenger service in 2017. 4.99 First, the low remaining contingency puts the project itself at risk of further delays. Previously, the project aimed to achieve its original schedule and recover delays through implementing numerous DRMs. Going forward, the insufficient contingency could severely limit the Project’s ability to quickly react to future unforeseen events and recover potential delays. 4.100 Second, in interviews with the Independent Experts, members of the PjT expressed the view that the uncertainty regarding the funding of the increased budget has created reservations among the contractors. The contractors are concerned how extensions to programmes due to the move of the opening date from 2015 to 2017 will be covered. 4.101 Third, cost escalation, i.e. the risk of inflation due to the prolonged time frame, might put the project further at risk. Every six months (in March and October) Government forecasts the inflation for future works contracts. The so-called money of the day forecast is the basis of price expectations for public bids. 4.102 Table 16 compares the October 2009 Government forecast of inflation in the public works projects with the actual figures for labour prices in civil engineering and building works. The growing divergence between forecast and actual is a symptom of the overheating construction sector in Hong Kong. Theoretically, in lump-sum contracts the Project is commercially protected against the unexpected inflation; however, the macro-economic development A-71 squeezes Project contractors and impacts the Project on non-lumpsum contracts. The pressures on contractors subsequently create risks for the Project in the form of potential increased volumes of claims by contractors to recover lost profits, further shortages of labour due to the inability to pay the necessary rates and even, in the extreme, insolvency or liquidity risk for some contractors - all factors which put the cost and schedule of the Project further at risk. Table 16. Comparison of forecasted and actual inflation 2009 2010 2011 2012 2013 2014 2015 2016 2017 MOD forecast 2% 2% 2% 2% 2% 3% 3% 3% 3% Actual inflation 0% 2% 7% 11% 11% 11% labour civils Actual inflation 1% 3% 5% 10% 11% 10% labour building Note: Actual figures for 2014 are June to June, other actual figures show increases from December to December Sources: HKSAR Financial Services and Treasury Bureau, HKSAR Census and Statistics Department Mitigation of key cost risks 4.103 To mitigate cost risk MTRCL is developing contract resolution strategies. The Independent Experts note that such strategies might give the contractors the needed financial commitment required to avoid an adverse impact to the Project’s progress. Acknowledging that such strategies might give the contractor the necessary financial commitment, the Independent Experts recommend that the commitment of contractors is further strengthened by tying payments to the successful achievement of key project milestones, key dates of the contract or target productivity rates. Moreover, such a commercial strategy could include incentive payments to contractors who meet their best achievable programme ahead of the PTC. 4.104 Additionally, the Independent Experts recommend that MTRCL establishes a regular communication channel with key contractors to reassure them about the financial situation of the Project and ensure their solid commitment to the 2017 opening date. 4.105 Lastly, the Independent Experts recommend that the HK$71.5 billion estimate continues to be re-evaluated taking into account the Independent Experts’ findings, and that extra funds, if likely to be needed, are A-72 mobilised now in order to prevent a vicious circle of further delays and cost increases resulting from budget uncertainty. Key reporting milestones to the Board 4.106 As mentioned above the commercial strategy might tie payments and incentives to specific achieved milestones and production rates. Moreover, once key milestones in each of the critical contracts are achieved, the risk position for the Project changes. Table 17 lists the key milestones, which reflect the point in time when the risk of XRL will change and the remaining risks could be reassessed. A-73 Table 17. Key reporting milestones to the Board Contract 823A 824 810A 810A T&C 823A 826 810A 826 E&M 824 810A 823A T&C T&C E&M E&M T&C E&M T&C T&C T&C Milestone Re-launching of both north and south U/T TBM Complete D&B excavation Achieve B3 diaphragm action in north top-down area Complete Central Stability System and South Stability System Complete South Stability System statutory inspections Complete south U/T TBM excavation Complete U/T and D/T TBM excavation Erect all roof trusses Degree-1 completion of TBM tunnels for track-laying Complete track-laying for mainline D/T Degree-1 completion of D&B tunnels for track-laying Complete B4 / OHVD and platform track-laying for the first 4 tracks Complete north U/T TBM excavation Energise OHL for pilot section Complete statutory inspections for ventilation buildings Complete track-laying for mainline U/T Energise OHL for dynamic testing of trains in mainline from Nam Cheong to Mai Po, both Up and Down Track Complete statutory inspections WKT Complete WKT track-laying Dynamic testing of Whole Line Dynamic Complete integration T&C with Mainland Commence Trial Running Date Nov 2014 Dec 2014 Mar 2015 Apr 2015 June 2015 Jul 2015 Aug 2015 Dec 2015 Dec 2015 Dec 2015 Feb 2016 Aug 2016 Sep 2016 Sep 2016 Oct 2016 Nov 2016 Jan 2017 Jan 2017 Jan 2017 Feb 2017 May 2017 Aug 2017 A-74 4.107 First, after commissioning and testing the north TBM at site 823A the start of the U/T drilling drive is scheduled for November 2014. At this date it will become clear what the schedule buffer is in the PTC. 4.108 Second, by December 2014 site 824 aims to complete the drill and blast tunnel connecting site 823 with 825. 4.109 Third, site 810A aims to achieve B3 diaphragm action in the north top-down area by March 2015. Once diaphragm action is achieved the concreting rate should increase and the civil construction risks at WKT will be significantly reduced. 4.110 Fourth, by June 2015 the statutory inspections of the stability systems at WKT are planned to be completed. This is the first important milestone for the commissioning of the XRL. 4.111 Fifth, by April 2015 the WKT site plans to have completed the Central Stability System and the South Stability System. These are critical works in order to begin the erection of the roof trusses. 4.112 Sixth, site 823A plans to complete the south U/T tunnel excavation by July 2015. 4.113 Seventh, the cross-boundary tunnels are scheduled to be completed by August 2015. Achieving this milestone will de-risk the tunnelling activities due to the difficult geological conditions in this area. 4.114 Eighth, the WKT roof trusses are planned to be erected by December 2015. The complex design of the WKT roof is one of the risks to completion. Once the roof trusses are erected the risk for the remainder of the WKT work will be significantly reduced. 4.115 Ninth, contract 826 plans to achieve Degree-1 completion of both tunnels by December 2015. Achieving this milestone marks the handover of the cross-boundary tunnel segment to E&M. 4.116 Tenth, the E&M team expects the completion of track-laying of the D/T mainline by December 2015. Achieving this milestone will reduce the remaining construction risks of the track works and testing can be expanded to the full D/T of the mainline. 4.117 Eleventh, by February 2016 all drill and blast tunnels are planned to achieve Degree-1 completion and to have been handed over to E&M. A-75 4.118 Twelfth, the first four tracks at level B4 of the WKT are expected to be completed by August 2016 and handed over to E&M. This milestone carries significant risks due to the sequential nature of construction and E&M activities. Achieving this milestone will allow the E&M team to commence their works towards testing and commissioning. 4.119 Thirteenth, it is planned to complete the north U/T tunnelling drive on contract 823A by September 2016. Achieving this milestone will significantly de-risk the tunnelling activities of the Project due to the low performance of the TBM on the north D/T drive. 4.120 Fourteenth, the pilot section of the track will be energised by September 2016. When this milestone is achieved testing of the line can commence. 4.121 Fifteenth, by October 2016 the statutory inspections for the ventilation buildings are planned to be completed as part of the commissioning process. 4.122 Sixteenth, the E&M team plans to complete track-laying for the U/T mainline by November 2016. Achieving this milestone will again significantly reduce the risk of not being able to commence full day1 operations on both tracks by 2017. 4.123 Seventeenth, the E&M team plans to energise the OHL by January 2017. This will allow the start of the dynamic testing between Nam Cheong to Mai Po in both the U/T and D/T tunnels. 4.124 Eighteenth, by January 2017 all statutory inspections of WKT are planned to be completed. Achieving this milestone will close out most of the risks of the construction programme of the WKT. 4.125 Nineteenth, the E&M team plans to complete all track-laying works by January 2017 when the WKT track-laying is planned to be completed. Achieving this milestone will result in further de-risking of the project as most of the track-related building works will be completed. 4.126 Twentieth, dynamic testing of the whole XRL is planned to be completed by February 2017. 4.127 Twenty-first, by May 2017 the integration testing with the Mainland section is planned to be completed. A-76 4.128 Twenty-second, by August 2017 full trial operations are planned to commence. Achieving this milestone will be the last step before commencement of full passenger operations in November 2017. Most uncertainties of the project should have been resolved by then. 4.129 Table 18 lists the KPIs that track the progress of the Project towards key construction milestones. Table 18. KPIs to track key construction milestones Contract 824 810A 810A 823A 826 810A 826 E&M 824 810A 823A E&M E&M Milestone Complete D&B excavation Achieve B3 diaphragm action in north top-down area Complete Central Stability System and South Stability System Complete south U/T TBM excavation Complete U/T and D/T TBM excavation Erect all roof trusses Degree-1 completion of TBM tunnels for track-laying Complete track-laying for mainline D/T Degree-1 completion of D&B tunnels for track-laying Complete B4 / OHVD and platform track-laying for the first 4 tracks Complete north U/T TBM excavation Complete track-laying for mainline U/T WKT track-laying completed KPI Excavation rate Concrete production Steel production Excavation rate Excavation rate Steel production (temporary and permanent) Concrete production Track-laying production Concrete production Concrete production Excavation rate Track-laying production Track-laying production A-77 Part V – Project management 5.1 The Independent Experts were asked to taking into account the causes of delay described in the IBC’s First Report and delay mitigation and delay recovery measures already put in place or developed by the PjT since January 2010, to recommend any enhancements they may see as appropriate to the project management of the Project (including reporting to stakeholders) in order to better ensure the completion of the Project in as timely and cost effective manner as possible. 5.2 In order to recommend enhancements to project management the Independent Experts analysed two questions: (1) Were the causes of the delay of the XRL identified by the MTRLC project information management system (PIMS)? (2) If so, why was the delay not reported in a more timely fashion than was the case? 5.3 The Independent Experts found that underlying the delay to the Project was an initial optimistic schedule of implementation. As discussed above, schedule overruns are common in comparable projects: 71% of the projects in the benchmark were delayed and the average schedule overrun of all projects in the benchmark was +43%. The benchmark shows that the hope that “everything will go according to plan” is common in large infrastructure projects. However, the benchmark also shows that most projects do not go according to plan due to the near certainty that unforeseen events will occur. 5.4 The above benchmark also shows that schedule and cost overruns in past MTRCL and KCRC projects were infrequent and minor compared with the international benchmark projects, which lends some justification to the optimistic schedule at the beginning, although it must be remembered that XRL was a different type of project than past MTRCL and KCRC projects, as described above. 5.5 The initial schedule for the Project, from the signing of the Entrustment Agreement and construction start in January 2010 to the planned start of service in 2015, was estimated at 5.8 years. In comparison, projects in a reference class of 85 similar projects actually took on average 7.0 years from construction start to opening of service, or 21% longer than the XRL construction schedule estimate. In the view of the Independent Experts, this shows that the XRL schedule was optimistic from the start, which left the Project with inadequate schedule contingencies to absorb unforeseen events. A-78 The Independent Experts consider the initial optimism and the lack of adequate schedule contingencies to be the root cause of the delay to the Project. 5.6 The problems with the initial schedule were aggravated by continuing optimism during construction. This optimism became apparent when key milestones were missed and assumed rates of production not met. Despite this challenging situation the PjT, in particular the PjD and GM, appeared to believe in the ability of DRMs to make up for lost time and get the schedule back on track. As a result, for several critical months the reality of a rapidly declining likelihood of meeting the original schedule appears not to have been properly recognised, despite warnings from within the PjT, and was therefore not communicated outside the Project organisation, leading to surprise and negative reactions when the inevitability of delay was finally communicated to the Board, Government and the public. 5.7 Among the contracts in the Project, 810A (WKT) is the largest, most complicated and most difficult to be constructed. The contract was made even more troublesome due to delays on adjacent contracts (810B and 811B), as a consequence of which award of 810A was not possible until October 18, 2011. The contract commenced on October 24, 2011, which left only 3 years and 10 months to complete the Whole of Works to be ready for passenger service by August 2015. This was an extremely optimistic, if not impossible, schedule from the outset, in the judgment of the Independent Experts. In interview, MTRCL staff have acknowledged that the logistics of the Project programme were always tight. Everything had to run very smoothly from day one if MTRCL was going to meet the original target completion date. MTRCL staff further explained that, unfortunately, even from day one there were issues, not least delays handing over land for the site. However, delays due to quality issues with so-called "steel couplers" also delayed the WKT construction showing that the programme was not progressing “very smoothly”. 5.8 Moreover, the Independent Experts understand that members of the PjT and the contractors increasingly questioned in late 2013 whether it would be possible to recover the delays to the Project programme through DRMs or in the face of the labour shortage. 5.9 Finally, the black rain storm on 30 March 2014 at the site of contract 823A and subsequent flooding of the “Fan-li-hua II” TBM seems to finally have led to a realisation within the whole PjT that the delays A-79 could not be recovered. However, the Independent Experts found that the flooding and the resulting problems with the TBM were not the main issues, although they were important. The main cause of the delays had already been incurred before the flooding and the most critical delay was related to the lower than planned production rates at contracts 810A, 823A and 826. In hindsight the PjD acknowledged this fact, singling out the WKT site as the key reason for the delay. 5.10 After the flooding, the existence of optimism, as described above, has been acknowledged by the PjD. The Independent Experts understand from its review of the documentary evidence, however, that the PjD pushed his team and the contractors as hard as possible to meet deadlines. 5.11 A further root cause for the delays to the Project, in addition to the optimistic schedule, was the sequential and highly interdependent construction works which had to be implemented in a very limited time span and confined space, in particular for contract 810A. 5.12 The effects of these root causes, in the view of the Independent Experts, was that the PjT, despite its good intentions and hard work on DRMs, was in reality given a very challenging task to achieve from the outset. 5.13 The facts about delays were openly reported and the PjT proactively attempted to recover the Project through DRMs. However, the Independent Experts identified the following reasons why the delay was not reported in time: i. None of the three reporting systems, the PjT status reporting, the project management system, and the internal and external auditors "raised the flag" of irrecoverable project delay; ii. The interpretation of facts and the responsibility for making the judgment as to whether the 2015 opening date was to be considered feasible ultimately rested with a single person, namely the PjD; and iii. This judgment call was made all the more difficult due to its binary nature, i.e. a “yes” or a “no” as to whether the schedule could still be recovered. 5.14 The XRL is managed using MTRCL’s Project Integrated Management System (PIMS). The PIMS was developed in 1992 and was first used at A-80 the Lantau and Airport Railway Project in 1992. Following the KCRCMTRCL merger in 2006/2007, the PIMS reflects the collection of best practices used at both organisations. The PIMS is continuously updated and improved through changes overseen by a special steering group that meets on a quarterly basis. 5.15 The PIMS is a readily accessible system. PIMS covers progress, cost, safety, environmental, and risk indicators. Project performance is measured by KPIs, which are summarised in a one-page dashboard, including traffic lights to focus attention. The dashboard is supplemented by detailed reports for each contract. The dashboard is primarily aimed at informing decision making at the project level. 5.16 Reporting to the CEO and Board happens through project status reports on a monthly basis. The reports are based on the PIMS KPIs. Following the recommendations in the First IBC Report in July 2014, the presentation of these reports was changed. Currently, the monthly report to the Board includes a one page safety programme, and a cost summary for all on-going projects, followed by project by project reporting of progress, issues, stakeholder management and costs including a Gantt chart 8 showing critical path performance and an Scurve chart showing overall project performance. 5.17 The aim of the PIMS is to provide a documented management system that meets international standards for quality, safety, environment, risk and asset management. The PIMS is ISO 9001 certified. The PIMS is implemented in MTRCL projects through the use of a series of project manuals and practice notes. Projects’ compliance with PIMS is audited through internal audits and self-quality audits. 5.18 The Independent Experts found that according to its users the PIMS has proven effective for past projects under the “Ownership Approach”. Specifically, the PIMS has been successfully used in all previous MTRCL projects since 1994. In 2008, MTRCL commissioned independent reviews of the internal controls framework and the PIMS. These reviews found the PIMS to be fit for purpose and to be complete, robust and comparable to best practice. 8 A Gantt chart is a bar chart typically used in project management to illustrate the schedule, subdivided into tasks. A-81 5.19 For XRL, two special “Compliance Manuals” were developed based on the entrustment agreements between MTRCL and Government. These two manuals set out the compliance actions against the obligations under the entrustment agreements. 5.20 The Independent Experts reviewed the past XRL dashboards and several status reports. The Independent Experts found that the first dashboard for the XRL was prepared in June 2011. 5.21 As described in Part III, the Independent Experts found that delays were openly reported and that the PjT proactively attempted to recover delays through DRMs. The Independent Experts further found that delays were indeed reported in the XRL performance KPIs and dashboards as specified in the PIMS. 5.22 However, the Independent Experts observed that in June and July 2011 the XRL PjT reported an amber traffic light for the programme. After this – from August 2011 to May 2014 – the XRL team consistently reported red programme traffic lights, 34 continuous red lights in total. The Independent Experts found that the project reporting openly communicated these delays for nearly three years. 5.23 In summary, three separate channels exist to escalate and manage arising issues for MTRCL projects, including the XRL: (1) the PIMS, (2) several layers of internal and external project audits and (3) status reporting by the PjT. Yet, the Independent Experts found that none of the three channels raised, in a timely fashion, the flag indicating that delays might become irrecoverable for the XRL. The Independent Experts assessed MTRCL project management with a focus, in particular, on what changes could be incorporated going forward to ensure that a flag is raised on a timely basis if delays are likely again to become irrecoverable. Why did the PIMS not raise the flag about the delay of the XRL? 5.24 The Independent Experts consider that the KPI-based PIMS can be more effective because the defined organisational procedure of “Setting, Reporting and Review of Key Performance Indicators” (PIMS/PN/01-1/A2) does not include a clear and cogent procedure for escalation of issues of non-performance beyond the PjT. 5.25 The defined organisational procedure requires that KPIs are reviewed and acted upon by the PjT: “PM, in conjunction with the project team, should discuss and review the project-level KPI performance against A-82 the targets and the effectiveness of the mitigation measures taken, if any, at the monthly Project Progress Meeting.” 5.26 In other words, the procedure did not require any performance shortfalls to be brought to the attention of higher levels of management, even if programme performance continuously fell short of targets, for nearly three years in the case of XRL. Because of this missing requirement in the PIMS, the Project Quality Audit did not find any non-compliance with reporting, even for the nearly three years when the programme was reporting red traffic lights. 5.27 The Independent Experts found, however, that the PIMS contains all the necessary information to identify when a project becomes irrecoverable. Recommendations are made below in Part VII “Reporting Processes and Presentation” regarding how to better utilise this information for improved reporting. 5.28 Moreover, the current criteria for the green-amber-red traffic lights are one-size-fits-all criteria for all MTRC projects. In Part VII, the Independent Experts recommend using a more project-specific approach that provides information regarding the probability of achieving project targets based on the schedule and cost contingencies of individual projects and informed by the data in the PIMS KPIs. Why did the project audits not raise the flag about the delay of the XRL? 5.29 Several levels of auditing, including MTRCL internal audits and audits by the Government-appointed M&V auditor, are in place for XRL. The internal audits have been conducted to ensure conformity to the MTRCL PIMS. The purpose of the internal audits is to confirm the consistent use of the PIMS and thus that project information is reported correctly and comprehensively. However, only a self-declaration process is in place to ensure compliance with the Compliance Manuals. 5.30 The audit structure was established through two decisions by the Board. First, on 3 August 2010 the Audit Committee made the decision that “In light of the increasing importance of new railway projects to the Company, it was agreed that the Projects Director would be invited to give a regular briefing to the Audit Committee in future meetings similar to the Operations Director and Property Director.” Subsequently, the Audit Committee decided on 24 November 2010 that “For new Hong Kong railway projects and to minimize duplication of work, IAD would conduct an audit on contracts administration and financial related matters only, while the A-83 Projects Division would focus on project quality assurance. IAD’s Half-yearly Report to the Audit Committee would include the audit summary report of the project assurance team.” 5.31 The remit of the IAD is outlined by the Audit Committee’s terms of reference as: “The Audit Committee is required to review, at least annually, the effectiveness of the Corporation’s financial controls, internal control and risk management systems and to report to the Board that such a review has been carried out. These controls and systems allow the Board to monitor the Corporation’s overall financial position.” 5.32 In short, internal audits are carried out by the IAD for all costrelevant processes, except quality assurance, which is audited through self-quality audits by each project under the guidelines of the project quality assurance and compliance section. The IAD and the project quality assurance and compliance section are independent from the Project Management Team. 5.33 The Independent Experts consider that project programme and project cost are two inseparable parts of project management and recommend that the auditing of them is done together and under the same body, with reports submitted to the CWC pertaining to its oversight function for capital works. 5.34 In the relevant period, the IAD once audited selected contracts of the West Island Line, South Island Line (East) and XRL projects. The audit report found no major instances of non-compliance. The report further suggested improvements to the speed of claims resolution process, and the consistency of reporting and KPIs used across projects. 5.35 The Independent Experts found no evidence that the review of the effectiveness of internal controls by the IAD and the Audit Committee had helped raise the flag of irrecoverable delays for the Project. In the view of the Independent Experts, the problem is twofold: (1) the IAD audits focussed on assuring that facts were neither misrepresented nor omitted but were transparently communicated from the front line up to the PjD; and (2) the Audit Committee meeting minutes suggest that while the status of the Project was communicated transparently, the effect of the delays was not recognised, because the Committee relied on the interpretation by the PjD of the achievability of the Project programme to completion. The A-84 Independent Experts understand that the Audit Committee did not receive any progress reporting apart from the PjD briefing. 5.36 For example, on 7 February 2012 the Audit Committee reviewed XRL progress and found that “XRL was slightly behind schedule […] [but the PjD] was still confident [in] a completion by August or September of 2015”. Similarly, on 14 August 2013: “In general all five new lines/extensions […] had been progressing in line as planned and project delivery was expected to be on time with some challenges though. Budget management was satisfactory and still under control notwithstanding the volume of claims in place […] there would still be many challenges to overcome […] Delay Recovery Measures were being taken by Management to maintain [the] programme.” 5.37 The Independent Experts recommend that the remit of the project audits is enlarged to incorporate the effectiveness and efficiency of control structures, including tools and procedures to identify when problems, e.g. schedule delays and cost overruns, are likely to be irrecoverable, and therefore may pose financial and reputational risk to a project and MTRCL. 5.38 Additionally, improved guidelines to assess project status might aid the Audit Committee in their role of challenging the PjT and PjD. For instance, the Independent Experts recommend that consecutively reported red lights for more than a quarter should trigger an alert to the Audit Committee that it should consider referring the performance of that project to the Board as a concern that the project programme is at risk of moving into irrecoverable delay. 5.39 In addition, XRL external audits are carried out regularly by the Government-appointed M&V auditor. The M&V audit reports, reviewed by the Independent Experts, demonstrated that the auditor challenged the project on aspects of technical designs as well as programming issues. In April 2013, the M&V list of critical issues included slow progress at contract 826 and warned of a delay of 1719 months. The M&V list of critical issues from August 2013 similarly estimated a delay of 9.5 months and posed the following question: “…what allowance has MTRCL made in its overall calculations for the critical delays at WKT and Contracts 820 and 826?” The response from MTRCL management was: “The current delays at WKT, 820 and 826 are being monitored closely and their impacts to the project is being assessed and reviewed.” The M&V audit of 12 September 2013 found that key dates were being missed A-85 and that track access had been impacted. The Independent Experts found no evidence that MTRCL’s responses did not satisfy the M&V auditor. 5.40 In sum, the M&V audits, as well as the 2013 Internal Quality Audits, Self Quality Audits and Technical Audits have found no evidence of significant non-compliance by the PjT with the respective project management systems. On the basis of the available evidence, the Independent Experts are reasonably satisfied by the audits that the PjT has followed the processes in the PIMS for the Project. 5.41 However, the Independent Experts recommend that when there is a continuous period in which a project reports red traffic lights this fact should be considered a new non–compliance event after a certain defined period, e.g. three months. The system should then require specific actions to be undertaken to rectify the situation before a specified deadline. In addition, the Independent Experts recommend that MTRCL considers enlarging the remit of the project audits to incorporate the effectiveness and efficiency of control structures, including tools and procedures to identify when problems, e.g. schedule delays and cost overruns, are likely to be irrecoverable, and therefore may pose financial and reputational risk to the relevant project and MTRCL. Why did the project status reporting by the PjT not raise the flag about the delay of the XRL? 5.42 As detailed above, the Independent Experts found that facts about delays were openly and correctly reported by the PjT and the team proactively attempted to recover the project through DRMs. 5.43 As regards reporting to Government, the Independent Experts learned in its interviews with MTRCL staff that Government was closely involved in problem-solving for the XRL delays. This happened through the Project’s bi-weekly meetings with Government and daily reporting on progress of the tunnels in Contract 826. Government supported this work through a designated team of engineers. Government was fully aware, for instance, of the delays in the cross-boundary tunnel contract and actively helped resolve the issues by working with the Mainland government. A-86 5.44 The Independent Experts also learned in its interviews that the different project teams on the XRL were not only meeting on a regular basis but also interacted daily and kept an active communication flow in the PjT. High risk areas (e.g. water inflow in contract 824, prefabrication of roof in contract 810A) were closely monitored. The Independent Experts were told of and saw protocols that demonstrated a high level of on-site supervision of contractors by MTRCL. 5.45 The Independent Experts conclude that although delays for each contract were factually reported, the interpretation of the consequences of the delays, and the responsibility for making the judgment as to whether the 2015 opening date would be feasible, ultimately rested with a single person only, namely the PjD. The impact of the delays on the overall schedule would have to be, and was ultimately, raised by the PjD based on all the reporting systems and outcomes available to him. 5.46 Moreover, the Independent Experts found that a contributing factor as to why the PjD did not report irrecoverability of the delay earlier is that the current PIMS and Compliance Manuals lack systematic indicators that would prescribe when the reporting of delays of a certain size and likelihood of irrecoverability must take place. Such a check would have assisted the PjD in making this key reporting decision. Instead, the current practice of reporting a delay as being beyond recoverability relies on intuitive judgment by the PjD aided by past experience, supported by the interpretations and experience of other members of the PjT. This judgment call is made all the more difficult due to its binary nature in the current practice of reporting, i.e. a “yes” or a “no” as to whether the schedule may still be recovered. Reversing the Project’s position that the planned opening was no longer feasible and that delays were irrecoverable was particularly challenging given stakeholder expectations. 5.47 In sum, the Independent Experts have not seen evidence that the PjT, including the PjD and GM, has not followed the systems and procedures of the PIMS and Compliance Manuals. The Independent Experts have seen evidence, however, that there is scope for improving these systems and procedures as regards systematic reporting on the effect of delays on the Project programme (see Part III above). A-87 5.48 The Independent Experts recommend that MTRCL develops a project status reporting system based on the probability of achievability pre-determined targets, such as completion or handover dates. The enhancements, which will be detailed in Parts VI and VII, can be made as part of MTRCL’s on-going improvements of the PIMS. Conclusions 5.49 In sum, the Independent Experts found that several layers of auditing and several reporting channels are in place to communicate and escalate issues in MTRCL projects. The Independent Experts have seen no evidence of non-compliance in the audit reports by the IAD, the project quality assurance and compliance section and the Government-appointed M&V auditor. Yet, none of these audits or reporting systems raised a flag that delays on the project were becoming irrecoverable. 5.50 The Independent Experts assessed MTRCL project management with a focus in particular on what changes could be incorporated to enhance the warning systems regarding the effects of delays going forward. The Independent Experts recommend enhancements to all three reporting channels. 5.51 First, the Independent Experts recommend enhancing the effectiveness of the programme KPIs through: i. Ensuring top management’s attention is drawn to areas where red traffic lights persist for a long duration; and ii. Tailoring special criteria for the two critical path contracts 810A and 823A, as well as for other high risk contracts, e.g. 826. 5.52 Second, the Independent Experts recommend enhancing the effectiveness of auditing through placing the audit of project cost and programme under one responsibility, with reports submitted to the CWC pertaining to its oversight function for capital works. A strong project auditing and assurance unit could provide senior management with an independent channel for reporting the status of projects and of issues of escalation. A-88 5.53 Third, the Independent Experts recommend enhancing the project status reporting through implementing an approach that reports how likely it is that a project is going to achieve its targets. Reporting whether project delays may be recovered or not should not rely solely on one or even a few persons’ judgment. The step change from considering delays recoverable to considering them irrecoverable is difficult and should be supported more objectively. 5.54 The enhancements for budget cost control and progress reporting are detailed in Parts VI and VII. 5.55 Lastly, in addition to the general issue of constantly and consistently improving the MTRCL project management and reporting systems to better support project delivery, including for the Project, the Independent Experts recommend that the Board carefully monitors the project management leadership situation through this transition period of change of PjD and CEO with a view to minimizing or eliminating any possible risk to the Project from the change in leadership. A-89 Part VI – Budget control 6.1 The Independent Experts were asked to: i. recommend any enhancements to budget control and reporting systems within the project management of the Project; and ii. recommend key reporting milestones in the future project programme for reports to the Board to facilitate monitoring of the actual Project budget against the proposed Project budget and anticipated eventual costs outturn. Recommendations for budget control and reporting systems 6.2 The cost control processes currently in place for MTRCL projects track cost development against the contract baseline (i.e. the Initial Contract Control Total). The overall cost risk position is regularly estimated by the PjT. The risk and contingency status are reported monthly to the GM, PjD and the Project Control Group (PCG). If a contract exceeds its defined cost control total a change request is made to the PCG. Approved change requests are funded from the available project contingencies. MTRCL has weekly PCG meetings, which ensure a fast process of approving or rejecting claims and change requests. 6.3 MTRCL’s system of estimating and controlling cost has been used with success in previous MTRCL projects. Only one of six past projects exceeded their budget: the Lantau Airport Railway by 6%. 6.4 In order for the MTRCL budget control system to become better capable of quickly detecting and hedging against such potential cost overruns, the Independent Experts recommend: i. The use of contingency draw down as a leading indicator to detect escalation earlier in the project cycle; ii. The use of reference class forecasting to establish appropriate contingencies for highly complex and uncertain projects; and iii. The establishment of a tiered contingency fund, in particular for projects funded by Government under the “concession approach.” A-90 Contingency draw down 6.5 The current process of cost controls tracks the cost implications of events that impact the Project schedule. The monthly Project reports include an overview of additional funding needs and project by project reports on the estimated final project cost, contingency analysis and cost to complete outlook. The project dashboard compares a project’s actual and planned payment schedule. Currently, no traffic light indicator is being used to report on Project cost. 6.6 The Independent Experts reviewed past XRL progress reports for the period from November 2011 to July 2014. The Independent Experts found that the actual monthly payments consistently lagged behind the planned payment schedule by an average of 4%. 6.7 However, actual payments were consistently ahead of the actual progress by on average 23%. This is unusual compared to industry practice, but members of Procurement explained in interviews that it is standard MTRCL practice and that it is done to ensure cash flow to contractors. The key benefit was described to the Independent Experts as increasing contract security, including retention, bonding and guarantees. This practice might lead to the loss of advance payments in the case of insolvency of a contractor. 6.8 Currently, cost forecasts are based on the changes requested and approved plus the contract baseline. For XRL Procurement has reported a most likely, a pessimistic and an optimistic scenario for the cost position of the Project. These scenarios are based on events that have caused entitlements and other risk areas and vary with the rate of approved claims. Following the First IBC Report, MTRCL has revised the reporting to the ExCom (and Board), which includes projections for risk and outturn cost and an assessment of the adequacy of contingencies. This should allow ExCom and Board to more easily detect potential inadequacies in the estimated outturn cost. 6.9 As discussed in Part II, the Independent Experts find that the key cost risks for the XRL are best reflected in the draw down of contingencies. The contingency funds are used to pay for the extension to the Project programme of moving the opening date from 2015 to 2017 and unforeseen delays to finish the whole of works. The Independent Experts recommend that this cost risk is closely monitored by including in the project summary the monthly A-91 contingency draw down and showing its accumulation against total and remaining contingencies. 6.10 The current practice of constructing a most likely, a pessimistic and an optimistic scenario for the cost position of projects follows the logic of earned value management systems, which make the assumption that current variations from the cost baseline are of a temporary nature. However, experience from large, complex projects, like XRL, show that cost variations on such projects are often systemic, caused by optimism in initial budgets. Thus scenarios for the most likely, optimistic, and pessimistic outturn cost should also incorporate a scenario where the actual historical volume of claims and variations is projected into the future. 6.11 For example, a trending indicator of the contingency draw down could indicate (1) a conservative estimate of the total outturn cost, provided contingencies are drawn down at a constant rate, (2) the probability of staying within the currently forecasted HK$71.5 billion outturn cost, i.e. a P-value based on actual outturn cost and actual cost overruns in a reference group of similar past projects 6.12 Additionally, project cost reporting should make it easier to establish whether a project is likely to overrun its budget and if so by how much. This could be achieved by including a cost performance index similar to the schedule performance index that is currently part of the progress reports. Other information to report could be the overall cost contingency position, major changes to the contingency position and a forecast of total outturn cost. 6.13 The Independent Experts were told in interviews with MTRCL staff that as the contingency fund depletes the risk of an adverse impact on the progress of the Project increases (and the faster the contingency fund depletes, the more quickly that risk increases). A lack of contingency might not only prohibit additional DRMs but might also incentivise contractors to operate at a consistent level of productivity which is too low to achieve timely completion of the Project instead of increasing productivity. The Independent Experts recommend that the Project contingency is replenished if the objective is to ensure the 2017 opening without further delay. A-92 Reference class forecasting 6.14 Reference class forecasting (RCF) is a new forecasting method aimed at identifying and eliminating optimism in projections. Research shows that both the frequency and scale of optimism is large for transport infrastructure forecasts in general and for forecasts of rail construction cost in particular. RCF achieves accuracy in projections by firmly basing them on actual performance in a reference class of comparable, completed projects and thereby bypassing optimism and increasing accuracy. RCF is described in more detail in Appendix 2. 6.15 For the XRL and other MTRCL projects, RCF would work in the following manner. After a bottom-up cost risk assessment has been produced the level of contingencies should be checked by taking the so-called “outside view”. This requires three steps: i. Identifying a relevant reference class of completed projects. The class must be broad enough to be statistically meaningful but narrow enough to be truly comparable with the specific project; ii. Establishing a probability distribution for the selected reference class for the variable of interest, e.g. risk of cost overruns. This requires access to credible, empirical data for a sufficient number of projects within the reference class to make statistically meaningful conclusions; and iii. Comparing the specific project with the reference class distribution in order to establish the most likely outcome for the specific project. 6.16 The key advantage of using reference class forecasting to establish the appropriate amount of contingencies is that the process is firmly grounded in empirical probability distributions of cost and schedule overruns of actually completed projects. It is crucial that contingencies be empirically based in this manner; otherwise there is a high risk of reintroducing optimism and bias in project preparation and decision making. 6.17 With the benefit of hindsight, a reference class forecast for the Project based on the projects used in the cost benchmarkings above would have suggested a contingency fund of +66% to reach a level of 80% budget certainty, that is a 20% risk of cost overruns. At a 50% level of certainty, i.e. a 50% risk of cost overruns the A-93 appropriate amount of contingencies would have been +30%. The 7% that were initially allocated to the Project and later replenished due to procurement savings were equivalent to only a 30% level of certainty (i.e. they carried a 70% risk of being exceeded). The later procurement savings increased the level of certainty for the original HK$65 billion budget to approximately 40%, i.e. it carried a 60% risk of being insufficient. Actual developments have borne out that the initial budget was underestimated, with contingencies that were too low. Most likely an RCF would have resulted in a budget, initially and presently, with more adequate contingencies. Tiered contingency fund 6.18 The Independent Experts recommend that, in line with industry practice, MTRCL establishes a tiered contingency management approach. Currently contingencies are solely managed by the PCG and change orders within the GM fund can be authorized by the Engineer’s Representatives. 6.19 International experience shows that a tiered contingency fund for publicly financed projects can be highly effective. The specific structure of the contingency fund depends on the characteristics and risks of each project and should be separately negotiated. An example of the international practice, adapted to the context of MTRCL “concession approach” projects, could be structured as follows, for example: i. Project holds the ICCT + 10%, subject to the authority of the PCG; ii. MTRCL holds contingencies up to the P40 level (i.e. contingencies sufficient to ensure that the budget will not overrun with a 40% level of certainty), subject to the authority of Government; and iii. Government holds the remaining contingencies up to the P80 or P90 level (i.e. contingencies sufficient that the budget will not overrun with a 80/90% level of certainty). 6.20 Tiered contingency management as described here helps to reassure contractors that funding is available to secure smooth project delivery. On the one hand the overall contingency position would become public knowledge. On the other hand the tiered approach signals that access to contingency funds is increasingly difficult. This helps to A-94 avoid the so-called "red meat" syndrome, where contingencies are used up simply because they are there and are common knowledge. A tiered contingency scheme that includes Government-held, or underwritten but not released, funds would alleviate concerns that future extensions to the Project programme (if any) and DRMs may be insufficiently funded or will be subject to politics. Recommendations for key reporting milestones 6.21 With regards to the reporting frequency, the Independent Experts find that the current practice of monthly updates to the contingency position and projected outturn cost is sufficient to steer the Project. 6.22 Consistent with the proposed project milestones (4.106), the key milestones for cost reporting are listed in Table 19. Moreover the key milestones listed in Table 19 reflect the point in time where the cost risk of the Project will change and the remaining cost risks could be reassessed. However, in the view of the Independent Experts, the Project has now a window of opportunity to secure funding. A-95 Table 19. Key reporting milestones to the Board Contract 823A 824 810A 810A T&C 823A 826 810A 826 E&M 824 810A 823A T&C T&C E&M E&M T&C E&M T&C T&C T&C Milestone Re-launching of both north and south U/T TBM Complete D&B excavation Achieve B3 diaphragm action in north top-down area Complete Central Stability System and South Stability System Complete South Stability System statutory inspections Complete south U/T TBM excavation Complete U/T and D/T TBM excavation Erect all roof trusses Degree-1 completion of TBM tunnels for track-laying Complete track-laying for mainline D/T Degree-1 completion of D&B tunnels for track-laying Complete B4 / OHVD and platform track-laying for the first 4 tracks Complete north U/T TBM excavation Energise OHL for pilot section Complete statutory inspections for ventilation buildings Complete track-laying for mainline U/T Energise OHL for dynamic testing of trains in mainline from Nam Cheong to Mai Po, both Up and Down Track Complete statutory inspections WKT Complete WKT track-laying Dynamic testing of Whole Line Dynamic Complete integration T&C with Mainland Commence Trial Running Date Nov 2014 Dec 2014 Mar 2015 Apr 2015 June 2015 Jul 2015 Aug 2015 Dec 2015 Dec 2015 Dec 2015 Feb 2016 Aug 2016 Sep 2016 Sep 2016 Oct 2016 Nov 2016 Jan 2017 Jan 2017 Jan 2017 Feb 2017 May 2017 Aug 2017 A-96 Part VII – Reporting processes and presentation 7.1 The Independent Experts were asked to: i. recommend any enhancements to the systems within the project management structure for the monitoring and reporting of: (1) progress toward the key reporting milestones and (2) developments that may affect the seriousness of risks identified already by the PjT or subsequently by the Independent Experts; and ii. recommend any enhancements to the reporting of progress of the Project in order to better ensure transparency and accessibility for all stakeholders or recipients of respective reports. This may include recommendations regarding the presentation and format of such reports as well as their content. 7.2 The Independent Experts reviewed the reporting processes of the Project as defined by the MTRCL Corporate Structure, the PIMS Manuals and the Compliance Manuals referring to the entrustment agreements. 7.3 The Independent Experts further reviewed the actual contents of these reports and meetings in order to identify potential enhancements. Based on these analyses the Independent Experts recommend: i. Enhancements to the monitoring and reporting; and ii. Enhancements to the presentation of factual data to ensure transparency. Existing reporting processes 7.4 Table 20 summarises all reporting and meetings required for the PjD, PjT and GM. A-97 Table 20. Summary of regular XLR meetings (in attendance, () on invitation) Meeting Board Meeting Meeting Participants MTRCL FreGovern Conquency PjD GM PM CM SConE -ment tractor Internal Meetings When   required Executive Weekly  Committee Meeting Audit Committee Quarterly  Meeting Project Control Group Weekly   XRL Progress Meeting Monthly    PjD Project Weekly Weekly    Communication Meeting PjD Senior Project Management Review Bi-weekly    Meeting XRL Project Risk Monthly   Review Meeting XRL Stakeholder Engagement Steering Monthly   Committee Meeting GM Weekly Weekly Communication   Meeting GM Weekly Weekly   Communication Meeting (Civil) PM Communication Weekly  Meeting CM Internal Varies Meetings External Meetings Project Supervision Monthly   () Committee (PSC) RDO Coordination Monthly   Meeting Contract Review Monthly  Meeting               A-98 Meeting RDO/MTRCL Public Relations Task Group Meeting XRL Monthly Design/Construction Cost Meeting Senior Management Review Meeting CM Meeting Site Safety and Environmental Committee Meeting SConE Meeting Meeting Participants MTRCL FreGovern Conquency PjD GM PM CM SConE -ment tractor Monthly   Monthly   Quarterly    ()  Monthly    Monthly      Bi-weekly/ Monthly 7.5 These regular reports and meetings are arranged in a bottom-up sequence starting with the GM/XRL weekly communication meeting on Monday followed by the PjD weekly communication meeting on Tuesday. The reporting is then taken to the corporate level with the ExCom meeting chaired by the CEO on Thursday morning and the PCG meeting on Thursday afternoon. 7.6 The weekly PCG meeting also includes representatives from the HyD/RDO. Additionally, several monthly meetings are held between MTRCL and Government. Those meetings are: 7.7 i. Monthly PSC meeting: chaired by DHy; ii. Monthly Project Co-ordination Meeting: chaired by an Assistant Director of the HyD; and iii. Monthly Contract Review Meetings: chaired by the Chief Engineer of RDO. The Independent Experts reviewed the reports presented in these meetings and the meeting minutes. The Independent Experts found that detailed progress, cost, contractual, environmental and safety information were presented. The Independent Experts also found that these meetings were attended by all levels of Project staff and A-99 stakeholders. However, in the judgment of the Independent Experts, presentations of progress in the Project fell short of providing participants the opportunity to make effective judgments about the extent of delays and their recoverability through proposed DRMs. 7.8 In order to judge whether or not proposed DRMs are likely to recover a delay and bring the project back to its original plan, an extensive programme review must be done. Such reviews quantitatively analyse schedule and resource impacts of all events that have happened and all proposed DRMs. However, in the case of XRL new events continuously happened at the sites, further aggravated by labour shortages. As a result, for example, the MOR programme took almost a year to confirm. In the view of the Independent Experts, this shows that for a complex project such as XRL a full project revision is not easily completed, which creates the need for alternative ways to inform decision making of project and corporate management. 7.9 To address this need, the Independent Experts propose a simpler approach based on the programme information reported in the project dashboard. Data from the PIMS can be used to assess the work required to recover project delays and the probability of this increased workload being implemented. Thus, the data give an indication of how likely it is that the delays will be recovered. Enhancements to KPIs and the risk register Schedule Performance Index 7.10 The Independent Experts found that the interpretation of delays can be informed by the automated reporting system. The actual programme progress and the planned programme progress can be used to calculate a Schedule Performance Index (SPI), for the programme at a certain stage, as: SPI = Actual Progress / Planned Progress. 7.11 Figure 7 shows the SPI from November 2011 to July 2014 against the 2015 opening date. The graph shows that Project has gradually deteriorated from November 2011 until September 2013 when the performance index reached the lowest point of 0.65. The curve shows that at the lowest point the SPI for the civil work reached 0.6. The graph also shows that E&M finished most of the design work in July 2013 and subsequently ran out of work sites to achieve progress. A-100 Figure 7. Schedule Performance Index (SPI) of the XRL 7.12 In the view of the Independent Experts, if this information had been included in past reporting meetings it would have been clear to decision makers that: (1) despite the large number of DRMs being implemented, their effect on Project progress was minimal; and (2) the productivity of civil contracts started to improve slightly after October 2013 but at the same time the E&M contracts significantly deteriorated due to lack of work spaces. Schedule Recovery Index 7.13 An alternative way of reporting the recoverability of delays is to estimate the productivity increase that needs to be achieved and sustained for the remainder of the project in order to meet the opening date. This Schedule Recovery Index (SRI) is calculated as: SRI = (Planned Progress - Actual Progress) / (1 – Actual Progress) 7.14 Figure 8 depicts the required productivity increase for the Project between November 2011 and July 2014 to achieve the 2015 opening. 7.15 In the view of the Independent Experts, a productivity improvement of more than 30% is highly unlikely. Achieving and sustaining a 30% improvement is equivalent to working 3 hours overtime per 10 hour work day. The historic data in Figure 8 shows that after April 2013 the Project needed to achieve a more than 30% productivity increase to meet the 2015 opening date. By the end of 2013 the required A-101 increase in productivity was higher than 60%. It is the view of the Independent Experts that achieving such an increase would have been almost impossible. 7.16 Consequently, if decision makers had been presented with information like that above, they would, in the view of the Independent Experts, have concluded no later than the second half of 2013 that the Project was irrecoverable. Figure 8. Schedule Recovery Index (SRI) of the XRL Enhancement to the programme KPI 7.17 As discussed above, the Independent Experts found that the Project reported red lights for a prolonged period in time. This indicates that either the KPI definition was flawed or appropriate action was not taken. 7.18 According to the KPI definition, a red light indicates a delay of more than eight weeks. The buffer in the original baseline was twelve weeks. The Independent Experts recommend tailoring the definition of traffic lights to the specific cost and schedule contingencies in the current baseline of projects rather than using a consistent definition across all projects. 7.19 The Independent Experts further recommend that clear rules are defined that a project cannot report more than a given number of consecutive red lights (depending on the overall length of the project) before triggering a review of the project. The outcome of the review A-102 should be a re-baselining of the project and an actionable turn-around plan, in the view of the Independent Experts. 7.20 Consecutive red lights must be linked to a clear plan of action with clear accountabilities and deadlines to bring project performance back on track. In other words, the PjT and the contractors must have responsibility to demonstrably recover performance by a set deadline. Suggested DRMs need to clearly specify by when they are going to be met and how much they will improve performance, so that the PjT can select the appropriate steps for short and long term recovery. The decision as to whether DRMs are sufficient needs to be driven by data. 7.21 Following the First IBC Report, MTRCL has taken steps to improve the reporting process. The revised reports now include Gantt charts indicating the critical path and slippage of contracts against the last approved baseline. 7.22 Moreover, changes to the reporting process have been made. Project status reports now require sign off by the Chief Programming Engineer. This adjustment was made to use the role of the Chief Programming Engineer to critically challenge the PjT. However, the Independent Experts find that this measure, while working temporarily, might not prevent future projects from ambitious assessments as to how achievable plans may be. The Independent Experts therefore recommend that MTRCL strengthens an independent channel to assure projects as recommended in Part V. 7.23 The Independent Experts also recommend that MTRCL incorporates indicators similar to the SPI and SRI in its new form of reporting. MTRCL should adopt a probabilistic approach in the reporting, which can be quickly updated when events are continuously happening. Risk Register 7.24 The Independent Experts also reviewed the risk registers of the Project and its contracts. In the view of the Independent Experts, the risk register is fairly comprehensive and the regular risk reviews focus the PjT on taking proactive steps to ensure that safety, quality, schedule and cost targets are being met in the short and long-term outlook. 7.25 The overall project reporting should include a high-level summary of key risks. To aid understanding of the risk register. The register could A-103 either be structured in sections for each goal or, to avoid duplication, the risk register could include a separate rating for the impact on the key goals, namely project safety, quality, schedule and cost. A summary depiction included in the high-level project report would then aid decision makers in showing which goals are most at risk and which areas of the Project to focus on in order to challenge the PjT. This would strengthen the link between risk reporting and project performance reports. Senior participation 7.26 The First IBC Report concluded that: “the failure of the PjD to communicate the likelihood of delay had deprived the Corporation of the opportunity to manage the situation in which it found itself on 12 April and the following days in the best possible way.” 7.27 Subsequently, on 21 August 2014, the Board announced the formation of two new committees: the CWC and the Risk Committee. Both committees are intended to provide oversight “to ensure the works of MTR Corporation management continue to meet high standards and quality.” 7.28 The Independent Experts consider that the two new committees provide channels for the Board to identify whether projects go through periods of difficulty and provide the project with the Board’s support and perspective. 7.29 In the view of the Independent Experts, the work of these two new committees is best informed by existing reporting. The Independent Experts do not see the need to create new reports and/or presentations besides the existing ones as enhanced by the recommendations above. 7.30 Moreover, the Independent Experts recommend that the committee members invite contractors’ senior management and Government officials to have regular informal meetings to enhance teamwork and the capacity for problem resolution at the most senior levels. A-104 Recommendations to ensure transparency and accessibility 7.31 The June 2013 Internal Audit Report identified that the approval process for revisions to contracts’ completion obligations should be accelerated to ensure that the Project is transparent about the current level of performance and that delays are not hidden in lengthy approval processes for claims. The Independent Experts recommend that IAD continues to monitor projects to ensure open and transparent reporting. 7.32 The Independent Experts further recommend that performance against the safety, quality, schedule and cost goals of the overall Project and each contract should be communicated openly within the PjT and upwards, including traffic light assessments. One positive side effect of open and visual communication between contracts is to create a competitive environment where contracts aspire not to be the worst performing. A-105 Part VIII – Recommendations 8.1 This part summarises the recommendations made above by the Independent Experts. The summary is made under four main headings: (1) recommendations to de-risk critical path contracts, (2) recommendations to enhance budget control, (3) recommendations to enhance reporting processes and presentation and (4) recommendations for key reporting milestones. Recommendations to de-risk critical path contracts Contract 810A 8.2 The Independent Experts found that, although the PTC has defined a feasible plan for the construction of 810A, the programme remains very tight and critical, with little slack. The Independent Experts make the following three recommendations: i. The site team of MTRCL and the contractor should work closely and cooperatively as “one team” to face the challenges and remain effective and efficient; ii. Corporate-level attention and support ought to be given to the key tasks of the contract, aimed at achieving the key dates. The best talent and resources of MTRCL and the contractors should be allocated to the contract and all parties should be firmly committed to the opening date. MTRCL Senior Management should invite Senior Management of the contractors to jointly commit the best resources to the site and to reconfirm their commitment to achieving the key milestones; and iii. An effective contingency plan should be prepared and ready in case the worst-case scenario develops at WKT, including the involvement of all relevant experience and knowledge of the Civil, E&M and Operations teams. Contract 823A 8.3 The Independent Experts found that the achievement of the key dates of contract 823A mostly relies on the performance of the TBMs. To minimise the risk of under-performance of the TBMs and achieve the date, the Independent Experts recommend that the chairman and/or CEO of MTRCL contact the Chairman and/or CEO of the Contractor and the TBM Manufacturer to ensure that there is commitment and A-106 pressure from the most senior levels of management to keep the TBMs running with minimal delay until completion. Contract 826 8.4 The Independent Experts found that, for now, contract 826 is no longer on the critical path. The Independent Experts make the following three recommendations: i. Although sufficient buffers are available to contract 826 for now, the 826 team should continuously exercise its diligence to prevent this contract from becoming critical again; ii. The PjT should diligently monitor contract 826, in particular the time needed for repairs and achieved production rates; and iii. The current unexpected cutter head damage of the D/T TMB should be recovered as soon as possible. Testing and commissioning and trial operation 8.5 The Independent Experts have examined the fallback plans and measures and judged them to be a reasonable and effective way to deal with possible further delays. The Independent Experts recommend that: i. Once the definite concreting plan for WKT is in place, the E&M team should work with the 810A team to reach a contingency plan for the worst-case scenario of delay in the handover to track-laying in order to try to ensure that that delay will have no negative impact on meeting the opening date for passenger service at the end of 2017; ii. The Operations Division’s roadmap require resolution of ten “very challenging” items that are not fully under the control of MTRCL. Resolving them requires continuous dialogue, effective coordination and reasonable alignment of interests. The Independent Experts recommend that actions should not be relaxed due to the delay of the opening date to 2017. A-107 Recommendations to enhance budget control 8.6 8.7 The Independent Experts make two recommendations for the commercial strategy, currently being investigated by MTRCL, in particular: i. The commercial strategy should assures financial commitment to the contractors. Such commitment should further be strengthened by tying payments to the successful achievement of key project milestones, key dates of the contract or target productivity rates. Moreover a commercial strategy should include incentives for early achievement of milestones; and ii. MTRCL should establish a regular, top-management communication channel with key contractors to reassure them about the financial situation of the project and to ensure their solid commitment to the opening date 2017. The Independent Experts found in its top-down and bottom-up assessment that, in its view, the budget is most likely going to be exceeded. The Independent Experts make the following recommendations to enhance the budget control: i. The Independent Experts found that the key cost risk for XRL is best reflected in the draw down of contingencies. The contingency funds are used to pay for the extension to the programme of moving the opening date from 2015 to 2017 and unforeseen delays to finish the whole of works. The Independent Experts recommend that this cost risk is closely monitored by including in the project summary the monthly contingency draw down and showing its accumulation against total and remaining contingencies; ii. The current MTRCL practice of constructing the most likely, pessimistic and optimistic scenario for the cost position of projects follows the logic of earned value management systems, based on the assumption that current variations from the cost baseline are of a temporary nature. However, experience from large, complex projects, like XRL, shows that the cost variations on such projects are often systemic, caused by optimism in initial budgets. Thus scenarios for the most likely, optimistic, and pessimistic outturn cost should also incorporate a scenario which assumes that the past volume of claims and variations will continue into the future; A-108 iii. A trending indicator of the contingency draw down could indicate: (1) the most likely total outturn cost, provided contingencies are drawn down at a constant rate, (2) the probability of staying within the currently forecasted HK$71.5 billion outturn cost, i.e. a P-value based on actual outturn cost and actual cost overruns in a reference group of similar past projects; iv. Project cost reporting should make it easier to establish whether a project is likely to overrun its budget and if so by how much. This can be achieved by including a cost performance index similar to the schedule performance index which is currently part of the progress reports. Other information to report could be the overall cost contingency position, major changes to the contingency position, and a forecast of the total outturn cost estimate; v. The HK$71.5 billion project estimate should be reviewed in the light of the Independent Experts’ verification; and vi. The Independent Experts recommend that, in line with international practice, MTRCL establishes a tiered contingency management approach. Currently contingencies are solely managed by the PCG and change orders within the GM fund can be authorized by the Engineer’s Representatives. Such an approach could, for example, adapting international practice to the context of MTRCL’s “concession approach” projects, be structured as follows: a. Projects holds the ICCT + 10%, subject to the authority of the PCG; b. MTRCL holds contingencies up to the P40 level (i.e. contingencies sufficient to ensure that the budget will not overrun with a 40% level of certainty), subject to the authority of Government; and c. Government holds the remaining contingencies up to the P80 or P90 level (i.e. contingencies sufficient that the budget will not overrun with a 80/90% level of certainty). A-109 Recommendations to enhance reporting processes and presentation 8.8 The Independent Experts reviewed the three channels in place in the XRL to escalate issues. The Independent Experts made recommendations to enhance the (1) Project Information Management System (PIMS), (2) project auditing and (3) project status reporting. Recommended enhancements to the PIMS 8.9 The Independent Experts consider that the KPI-based PIMS can be made more effective. The Independent Experts make three recommendations to enhance the PIMS: i. The current criteria for the green-amber-red traffic lights are onesize-fit-all criteria for all MTRC projects. The Independent Experts recommend that the traffic light alert system is based upon planned schedule and cost contingencies; ii. The Independent Experts recommend using an approach which more clearly reports on the probability of achieving key dates and milestones; and iii. Strengthen the effectiveness of the programme KPI through: a. Ensuring top management’s attention is drawn to areas where red traffic lights persist for a long duration; b. Tailoring special criteria for the two critical path contracts 810A and 823A, as well as for other high-risk contracts, e.g. 826. Recommended enhancements to project auditing 8.10 The Independent Experts recommend establishing strong project auditing and assurance in order to provide senior management with an independent channel for receiving reports on the true status of projects and to escalate issues to senior management. The Independent Experts make four recommendations for enhancing the project auditing in this way: i. The Independent Experts consider the project programme and project cost to be two inseparable parts of project management. They recommend that the auditing of these two functions is done together and under the same body, with reports submitted to the CWC pertaining to its oversight function for capital works; A-110 ii. The Independent Experts recommend that the remit of the project audits is enlarged to incorporate the effectiveness and efficiency of control structures, including tools and procedures to identify when problems, e.g. schedule delays and cost overruns, are likely to be irrecoverable, and therefore may pose financial and reputational risk to the Project and MTRCL; iii. Additionally, improved guidelines to assess project status might aid the Audit Committee in its role of challenging the PjT and PjD. For instance, the Independent Experts recommend that where red lights are reported consecutively for more than a quarter in relation to any project’s performance, that event should trigger an alert to the Audit Committee that it should consider referring the project’s performance as an area of concern to the Board; and iv. The Independent Experts recommend that the continuous display of red traffic lights, following current procedures, be considered a new “non–compliance” event after a certain time period, e.g. three months, that requires specific actions to be undertaken to rectify the situation before a specified deadline. Recommended enhancements to project status reporting 8.11 The Independent Experts recommend strengthening the project status reporting as follows: i. MTRCL should develop and adopt measures to ensure that reporting whether a project delay can be recovered does not solely rely on one or a few persons’ judgment. The Independent Experts recommend that this can be achieved through strengthening the audit channel or establishing guidelines when issues need to be escalated; ii. The step change from delays being recoverable to being irrecoverable is difficult to make. This decision can be better supported by a reporting system that shows the probability of achieving key dates and milestones. Thus, the Independent Experts recommend implementing a reporting system that shows more clearly the probability of achieving pre-defined milestones. The Independent Experts gave examples of probability-based KPIs above, e.g. to track: a. How likely it is that the target opening date is going to A-111 be met, given a project's current performance, by using the schedule performance index; or b. How likely it is that DRMs are going to recover delays, by using the schedule recovery index. iii. The Independent Experts also observe that the Project reported red lights for a prolonged period in time. This indicates that either the KPI definition was flawed or that appropriate action was not taken. The Independent Experts recommend tailoring the definition for how traffic lights are set to the specific cost and schedule contingencies in the current project programme rather than using the same definition across all projects. iv. The Independent Experts further recommend defining clear rules that a project cannot report more than a given number of consecutive red lights without triggering an alert or response in the project management system. The relevant number of red lights may be varied depending upon the overall length of the project. Reporting several red lights in a row should trigger a process to review the project with the outcome of putting a project turn-around plan into action or re-baselining the project. v. Red lights in a project need to be linked to a clear plan of actions with clear accountabilities and deadlines to bring project performance back on track. The Independent Experts recommend establishing reporting that holds the PjT and the contractors accountable to demonstrably recover performance by a set deadline. Suggested DRMs need to clearly specify by when they are going to improve performance and by how much. The judgment whether the DRMs are sufficient and whether performance has improved needs to be driven by data. vi. The Independent Experts observe that changes to the reporting process have been made. Project status reports now require sign-off by the Chief Programming Engineer. This adjustment was made to use the role of the Chief Programming Engineer to critically challenge the PjT. However, the Independent Experts find that this measure, while temporarily working, might not prevent future projects from ambitious assessments as to how achievable plans may be. The Independent Experts recommend that MTRCL should strengthen an independent channel to assure projects as outlined above. A-112 vii. The Independent Experts found that the risk register is currently not part of the project dashboard. The overall project reporting should include a high-level summary of key risks. To better link the risk register with the KPI reporting the Independent Experts recommend that risks are rated by their likely impact on key project goals, namely project safety, quality, schedule and cost. If a summary depiction of that risk matrix is then included in the high-level project report, that should aid decision makers in showing which goals are most at risk and which area of the project to focus on in order to challenge the relevant project team. viii. The Independent Experts consider that the two new Board committees established to oversee MTRCL projects (the CWC and Risk Committee) could provide channels for the Board to identify when projects go through periods of difficulty and when to provide any such project with the Board’s support, assistance and influence. In the view of the Independent Experts, the work of these two new committees is best informed by existing reporting. The Independent Experts do not see the need to create new reports and/or presentations besides the existing ones (as enhanced according to the Independent Experts’ recommendations above). ix. The Independent Experts recommend that the committee members invite contractors’ senior management and Government officials to have regular informal meetings to enhance the teamwork and resolution of problems beginning at the top level. x. Regarding the transparency of potential delays, the Independent Experts recommend that IAD continues to monitor that delays are not hidden in lengthy approval processes for claims. IAD should also continue to ensure that projects continue to report openly, transparently, and factually. xi. The Independent Experts further recommend that performance against the safety, quality, schedule and cost goals of the overall project and each contract should be communicated openly within the PjT and upwards, including traffic light assessments. 8.12 In addition to the above recommendations, the Independent Experts recommend that the Board carefully monitors the project management leadership situation through this transition period of change of PjD and A-113 CEO with a view to minimising or eliminating any risk to the Project from the change in leadership. Recommendations for key reporting milestones 8.13 The Independent Experts reviewed the PTC and the milestones of the XRL. The Independent Experts have identified key milestones, which reflect the point in time where the schedule and cost risks of XRL will change, and KPIs that track the progress of the Project towards these milestones. The Independent Experts recommend that these milestones are closely monitored and once they are achieved, the remaining risks may be reassessed. Table 21 lists the milestones, Table 22 lists the KPIs. A-114 Table 21. Key reporting milestones to the Board Contract 823A 824 810A 810A T&C 823A 826 810A 826 E&M 824 810A 823A T&C T&C E&M E&M T&C E&M T&C T&C T&C Milestone Re-launching of both north and south U/T TBM Complete D&B excavation Achieve B3 diaphragm action in north top-down area Complete Central Stability System and South Stability System Complete South Stability System statutory inspections Complete south U/T TBM excavation Complete U/T and D/T TBM excavation Erect all roof trusses Degree-1 completion of TBM tunnels for track-laying Complete track-laying for mainline D/T Degree-1 completion of D&B tunnels for track-laying Complete B4 / OHVD and platform track-laying for the first 4 tracks Complete north U/T TBM excavation Energise OHL for pilot section Complete statutory inspections for ventilation buildings Complete track-laying for mainline U/T Energise OHL for dynamic testing of trains in mainline from Nam Cheong to Mai Po, both Up and Down Track Complete statutory inspections WKT Complete WKT track-laying Dynamic testing of Whole Line Dynamic Complete integration T&C with Mainland Commence Trial Running Date Nov 2014 Dec 2014 Mar 2015 Apr 2015 June 2015 Jul 2015 Aug 2015 Dec 2015 Dec 2015 Dec 2015 Feb 2016 Aug 2016 Sep 2016 Sep 2016 Oct 2016 Nov 2016 Jan 2017 Jan 2017 Jan 2017 Feb 2017 May 2017 Aug 2017 A-115 Table 22. KPIs to track key milestones Contract 824 810A 810A 823A 826 810A 826 E&M 824 810A 823A E&M E&M Milestone Complete D&B excavation Achieve B3 diaphragm action in north top-down area Complete Central Stability System and South Stability System Complete south U/T TBM excavation Complete U/T and D/T TBM excavation Erect all roof trusses Degree-1 completion of TBM tunnels for track-laying Complete track-laying for mainline D/T Degree-1 completion of D&B tunnels for track-laying Complete B4 / OHVD and platform track-laying for the first 4 tracks Complete north U/T TBM excavation Complete track-laying for mainline U/T WKT track-laying completed KPI Excavation rate Concrete production Steel production Excavation rate Excavation rate Steel production (temporary and permanent) Concrete production Track-laying production Concrete production Concrete production Excavation rate Track-laying production Track-laying production A-116 Appendix 1 IBC members, appointed Independent Experts, interviews and site visit dates Members of the IBC9 Prof. Frederick Ma Si Hang (Chairman) Dr. Dorothy Chan Yuen Tak Fai Mr. Edward S T Ho Mr. Alasdair Morrison Mr. Abraham Shek Lai-him Independent Experts appointed by the IBC Prof. Bent Flyvbjerg Prof. Tsung-Chung Kao Interviews and site visit dates of the Independent Experts 01 August 2014 XRL Terminus Management Team 01 August 2014 Projects Director 08 August 2014 Contract 823A 08 August 2014 Project Quality Assurance 12 August 2014 Operations 12 August 2014 GM XRL 15 August 2014 Project Quality Assurance 15 August 2014 E&M 15 August 2014 Programming 15 August 2014 WKT contracts 18 August 2014 Internal Audit Department 18 August 2014 Procurement 19 August 2014 Contract 826 19 August 2014 Project Quality Assurance 22 August 2014 Operations 25 August 2014 Procurement, Programming 04 September 2014 Project Quality Assurance 04 September 2014 E&M 05 September 2014 Internal Audit Department 12 September 2014 Programming 16 September 2014 Programming 18 September 2014 Projects Director 9 Mr. T. Brian Stevenson resigned from the IBC shortly following production of the First IBC Report due to conflicting time commitments. A-117 Appendix 2 Reference Class Forecasting A brief history of Reference Class Forecasting RCF is founded on theories of decision making under uncertainty, which won Daniel Kahneman the Nobel prize in economics in 2002 10 11 12. After professional debate in Harvard Business Review in 2003 13 14 , RCF was developed by Flyvbjerg and Cowi 15 into an applied method for forecasting in policy and management practice. , , , The first instance of RCF in practical policy and management was carried out in the UK in 2004 by Flyvbjerg and Cowi under the auspices of HM Treasury and the UK Department for Transport16. Today, RCF is required for all large transport infrastructure investments in the UK seeking funding from the UK Department for Transport and HM Treasury, and many UK transport projects have been submitted to RCF, including London's £15 billion Crossrail project. In the USA in 2005, the American Planning Association (APA) officially endorsed RCF in the following manner: “APA encourages planners to use reference class forecasting in addition to traditional methods as a way to improve accuracy. The reference class forecasting method is beneficial for non-routine projects such as stadiums, museums, exhibit centers, and other local one-off projects. Planners should never rely solely on civil engineering technology as a way to generate project forecasts.” 17 Interest in and application of RCF is quickly spreading in developed and emerging economies alike. 10 Kahneman, D. and Tversky, A., 1979. Prospect theory: An analysis of decisions under risk, Econometrica, 47, pp. 313–327. 11 Kahneman, D. and Tversky, A., 1979. Intuitive prediction: Biases and corrective procedures, in: S. Makridakis and S. C. Wheelwright (Eds.) Studies in the Management Sciences: Forecasting, p. 12, Amsterdam: North Holland. 12 Kahneman, D., 1994. New challenges to the rationality assumption, Journal of Institutional and Theoretical Economics, 150, pp. 18–36. 13 Lovallo, D. et al., 2003. Delusions of success. Harvard Business Review, 81(7), pp.1–10. 14 Flyvbjerg, B., 2003. Delusions of success: Comment on Dan Lovallo and Daniel Kahneman. Harvard Business Review, 81(12), pp. 121-122. 15 Flyvbjerg, B. and COWI, 2004. Procedures for Dealing with Optimism Bias in Transport Planning: Guidance Document, The British Department for Transport: London. 16 Flyvbjerg, B., 2006. From Nobel Prize to Project Management: Getting Risks Right. Project Management Journal, 37(3), pp.5– 15. 17 American Planning Association, 2005. JAPA article calls on planners to help end inaccuracies in public project revenue forecasting. Available at http://www.planning.org/newsreleases/2005/ftp040705.htm A-118 Key concepts of Reference Class Forecasting , RCF is based on Kahneman and Tversky’s 18 19 finding that human judgment is generally optimistic due to overconfidence and insufficient regard to distributional information. Thus people will underestimate the costs, completion times, and risks of planned actions, whereas they will overestimate the benefits of the same actions. Such errors of judgment are shared by experts and laypeople alike, according to Kahneman and Tversky. Lovallo and Kahneman20 call such common behaviour the “planning fallacy.” They argue that this fallacy stems from actors taking an "inside view" focusing on the constituents of the specific planned action rather than on the outcomes of similar actions that have already been completed. At the root of the planning fallacy is a tendency for actors to see each new venture as unique. In fact, ventures are typically more similar than actors assume, even ventures that on the surface of things may appear quite different. For instance, planners may think that building a high-speed rail line and building a subway are two very different undertakings with little to gain from each other. In fact the two may be – and often are – quite similar in statistical terms, for example as regards the size of cost and time overruns or demand and revenue shortfalls. Thus the lessons from one project can be pooled with other similar projects and used as distributional information to statistically predict the outcome of another project. Kahneman and Tversky 21 argue that the prevalent tendency to underweigh or ignore distributional information is perhaps the major source of error in forecasting. Planners should therefore make every effort to frame the forecasting problem so as to facilitate utilizing all the distributional information that is available. 18 Kahneman, D. and Tversky, A., 1979. Prospect theory: An analysis of decisions under risk, Econometrica, 47, pp. 313–327. 19 Kahneman, D. and Tversky, A., 1979. Intuitive prediction: Biases and corrective procedures, in: S. Makridakis and S. C. Wheelwright (Eds.) Studies in the Management Sciences: Forecasting, p. 12, Amsterdam: North Holland. 20 Lovallo, D. et al., 2003. Delusions of success. Harvard Business Review, 81(7), pp.1–10. 21 Kahneman, D. and Tversky, A., 1979. Intuitive prediction: Biases and corrective procedures, in: S. Makridakis and S. C. Wheelwright (Eds.) Studies in the Management Sciences: Forecasting, p. 12, Amsterdam: North Holland. A-119 This may be considered the single most important piece of advice regarding how to increase accuracy in forecasting through improved methods 22. Using such distributional information from other ventures similar to that being forecasted is called taking an “outside view” and is the cure to the planning fallacy. RCF is a method for systematically taking an outside view on planned actions. RCF places particular emphasis on relevant distributional information because such information is most significant to the production of accurate forecasts. The three steps of Reference Class Forecasting i. Identification of a relevant reference class of past, similar projects. The class must be broad enough to be statistically meaningful but narrow enough to be truly comparable with the specific project. ii. Establishing a probability distribution for the selected reference class. This requires access to credible, empirical data for a sufficient number of projects within the reference class to make statistically meaningful conclusions. iii. Comparing the specific project with the reference class distribution, in order to establish the expected outcome for the specific project. Thus RCF does not try to forecast the specific uncertain events that will affect the particular project, but instead places the project in a statistical distribution of outcomes from the class of reference projects. In statisticians’ language, RCF consists of regressing forecasters' best guess toward the average of the reference class and expanding their estimate of credible interval toward the corresponding interval for the class. The comparative advantage of the outside view and RCF is most pronounced for non-routine projects, understood as projects that planners in a certain locale or organisation have rarely or never attempted before – like building new major infrastructure or catering to new types of demand. It is in the planning of such new efforts that biases in forecasts are likely to be largest. 22 Kahneman, D., 2011. Thinking, Fast and Slow. New York: Farrar, Strauss and Giroux. A-120 Choosing the right reference class of comparative past projects becomes more difficult when planners are forecasting initiatives for which precedents are not easily found, for instance the introduction of new and unfamiliar technologies. However, most projects are both non-routine locally and use well-known technologies. Such projects are, therefore, particularly likely to benefit from the outside view and RCF. A more full description of the theory and methodology of reference class forecasting was published in 2006 23. Adjusting forecasts for optimism For a cost forecast, when a relevant reference class of past, similar projects has been defined (step 1 above), the available statistical data on actual budget increases may be used to establish a probability distribution for this group of distinct transport schemes (step 2 above). Figure 9 below provides an example of the probability distribution for budget increases in the reference class of benchmarking projects from Part III. It furthermore illustrates the link between the observed ex-post cost increases for historical projects and the required up-lift for a new project to ensure that the probability of the final cost being higher than the initial budget plus the up-lift is less than a given threshold level. Figure 9 Overview of distribution of cost overruns and resulting uplifts required from benchmark in Part III If the new project is similar to the projects in the reference class (using similar technology and planning/governance regime) and the initial budget is established in a similar manner (not including budget contingencies 23 Flyvbjerg, B., 2006. From Nobel Prize to Project Management: Getting Risks Right. Project Management Journal, 37(3), pp.5–15. A-121 reflecting the risk of cost escalations above the level of contingencies used in the reference cases), the up-lift should be applied to the initial budget. The project team or other experts closely associated with the project may hold the inside view that the established initial budget is the best possible estimate of the average ex-post realised cost and that no up-lift is necessary. However, overwhelming data on past projects suggest that although this will be the case for a few individual projects it is not the typical case. Only for projects where risk analysis and mitigation has been applied beyond the level applied in the average project is there an argument for not applying the up-lift. In other words, assuming that the project in question is an average project, it should then be expected that the final budget - on average - will exceed the initial budget by the average budget increase. This also implies that there is a 50% chance of the budget increase being less than the average budget increase and a 50% chance of the budget increase being higher than the average budget increase. If it is not acceptable to decision makers that there is a 50% chance of the realized cost being higher than the budget (including up-lift), i.e. the revised budget needs to be more than 50% certain, the up-lift would need to be higher than the average budget increase. Figure 9 shows the necessary uplift to ensure that the level of certainty of a revised budget (including uplift) is above a given threshold (x%). For an organisation implementing a large portfolio of projects, the total realised budget increase across all projects can be expected to be close to the expected average. If the organisation has the ability to reallocate budgets between projects (portfolio management), application of an up-lift reflecting the average budget increase should therefore enable budget compliance on average. However, for the individual project organisation, a budget based on the initial budget plus an uplift reflecting the average budget increase will mean that there still is a 50% risk of the budget being inadequate. The right level of adjustment for optimism bias will therefore also be dependent on willingness to accept risk and the procedural issues related to budget revisions. A-122 Possible pitfalls of Reference Class Forecasting Budgets that are available to a project organisation often tend to get used. It is therefore important to recognise that the establishment of budgets which on average are more than adequate (as would be the case if uplifts are applied that reflect a higher percentile in the distribution than the 50% percentile) may have an incentive effect which works against tight cost control if the more than adequate budget is available (or perceived as being available) to the project organisation. This emphasises that the introduction of an outside view and associated uplifts should go hand in hand with contractual and process-oriented initiatives or governance structures, e.g. tiered contingency schemes, aimed at keeping cost down. Furthermore, there will typically be some degree of asymmetrical information between the project organisation and external observers. The introduction of uplifts may therefore in itself introduce an additional moral hazard risk in the principal-agent relationship between the project organisation (the agent) and the decision maker and funder of the project (the principal). This, and other issues regarding strategic behaviour and institutional set-ups should be given careful consideration in conjunction with a reference class forecast. Finally, a possible pitfall in employing the approach described above is that forecasters, when estimating the future cost of a specific project, would depart from the basic principles of reference class forecasting and would gradually return to the practices of conventional forecasting, with forecasters focusing on the details of the project at hand and attempting to forecast the specific events that would influence the future course of this project. Studies show that with the track record of conventional cost forecasting for rail, there is an 80-90% likelihood such behaviour would reintroduce optimism in forecasting. This pitfall may be avoided by consistently sticking to the method of reference class forecasting.