Showing posts with label Dysfunction in MTA’s $600 Million Subway Car Contract with Bombardier. Show all posts
Showing posts with label Dysfunction in MTA’s $600 Million Subway Car Contract with Bombardier. Show all posts

Tuesday, December 10, 2019

Comptroller Stringer Audit Reveals Delays, Defects, Dysfunction in MTA’s $600 Million Subway Car Contract with Bombardier


Bombardier consistently failed to produce acceptable work, meet project milestones and correct critical structural defects in a timely manner – resulting in a three-year delay
MTA failed to properly oversee the contract, waiting years before exercising critical enforcement authority
In midst of subway crisis, delays in delivery of new Bombardier cars increased cash-strapped MTA’s maintenance costs for existing subway cars by as much as $35 million
  New York City Comptroller Scott M. Stringer released an audit revealing that to date, the Metropolitan Transit Authority’s $600 million contract with Bombardier Transit Corporation to produce 300 subway cars is nearly three years behind schedule. The audit found that throughout the seven-year extended contract term Bombardier consistently failed to timely meet project milestones, comply with technical requirements, produce acceptable work, and promptly correct serious defects in critical structural components of the subway cars. In addition, the Comptroller’s audit shows that the MTA failed to adequately oversee Bombardier’s contract performance and timeliness and failed to complete required annual contractual evaluations to hold Bombardier accountable.
“The taxpayers who foot the bill for the MTA’s $600 million contract with Bombardier were promised new, state-of-the-art train cars to help modernize the city’s ailing, outdated transit system. Instead, commuters are dealing with more delays and more subway breakdowns as
we wait for trains that are three years behind schedule,” said New York City Comptroller Scott Stringer. “This audit reveals that MTA’s top-to-bottom mismanagement of Bombardier’s contract – from missed deadlines to bad welding to poor design – cost New Yorkers as much as $35 million to keep old, broken-down cars on the tracks long past their expiration date. By shedding light on these issues today, we are urging the MTA to exercise far greater accountability, oversight, and responsibility in their management these critical contracts. New Yorkers are paying for a 21st century subway system and deserve nothing less.”
In 2012, the MTA entered into a $599 million contract with Bombardier for the design, production and delivery of 300 R179 subway cars. The contract called for Bombardier to deliver 10 prototype cars to MTA by January 2015 and all 300 cars by January 2017. Bombardier missed both deadlines and by January 2017 had delivered only 18 cars of the 300 cars the contract called for. As of the date of this audit, delivery is still incomplete and is now 35 months behind the original contract schedule.
Comptroller Stringer’s audit found pervasive failures by Bombardier to timely meet contract requirements and deadlines, failures in its design and subcontracted manufacturing processes, and insufficient oversight by the MTA to minimize the delays and promptly enforce key contract requirements.
Although MTA and Bombardier negotiated revised deadlines, a $1.27 million payment reduction, and Bombardier’s agreement to deliver 18 additional subway cars at no additional cost, Bombardier again failed to meet the extended deadline of January 2019, delivering only 162 cars by that date. As of December 2019 – nearly three years after the original project completion date – MTA has reported that 298 of the 318 cars now required have been delivered, of which 278 are in service.
Failure to Comply with Contract and Poor Project Management
The Comptroller’s audit found that Bombardier failed to meet contract milestones and timely comply with other contract requirements in the following ways:
  • Substandard compliance during the design stage
  • Delays resulting from welding issues
  • Failure to properly oversee the overseas supplier and subcontractor that manufactured truck frame castings, critical structural components under the subway cars that support the cars and hold their wheels, motors, and other mechanical equipment in place.
  • Delays resulting from continuous failures of qualification tests
A table illustrating some of Bombardier’s missed project milestones and their cascading effect is below:
Comparison of Planned and Actual Completion Dates of Project Milestones
MilestoneDescriptionPlanned Completion DateActual Completion DateMonths Delay
AMobilization06/04/201206/04/20120
BManagement Plan, Master Schedule, Reliability Program, Safety System Program, QA Plan09/03/201209/27/20121
CSoftware QA Plan, Software Configuration Control Master Test Plan, Contractor Data Requirement08/03/201209/27/20122
DCopies of Subcontracts for Subsystems10/02/201207/31/2012-2
ECompletion of Preliminary Design Review02/04/201303/25/20131
FApproval of General & Section Category Drawings06/04/201311/20/20135
GApproval of Car Mock-Ups05/03/201311/20/20136
HApproval of Stress Analysis & Compression Test Reports12/04/201311/25/201523
IApproval of Production Design Drawings11/04/201301/30/201514
JApproval of Critical Design Review06/04/201302/26/20148
KCompletion & Acceptance of Inspection for First A&B Carbody Shells09/04/201311/25/201526
K1Completion & Acceptance of First Article Inspection For Truck Frame09/04/201308/26/2016See Below*
LCompletion & Acceptance of Inspection for First A&B Car Subsystems11/04/201305/26/201630
MApproval of Propulsion and Friction Brake Systems Performance Qualification Tests05/04/201510/24/201617
NReview of Complete A&B Cars07/04/201402/26/201619
ODelivery of First Two Five Car Units01/12/2015**11/22/201622
PCompletion & Acceptance of Car Qualification & Performance Tests on Units 1,2,5, and 608/04/2015See Below***
QCompletion of Acceptance Tests for Each Unit02/06/2017Ongoing
RSuccessful Completion of Acceptance Tests of the 16th , 40th Cars, as well as Units 1 and 201/04/2016See Below****
* Milestone K was spilt into K and K1 as per a Change Order executed on 11/24/2015.
**21 days of excused delay were granted due to pilot strike.
***Milestone P was split into P1, P2, and P3, due to the settlement agreement. Each had completion dates of 12/27/2017; 01/08/2018; 3/26/2019, respectively.
****Milestone R was split into R1, R2, and R3, due to the settlement agreement. Each had completion dates of 01/08/2018; 5/25/2018; 3/26/2019.
Even after the R179 trains were delivered and put in operation, cars had to be taken out of service on at least three different occasions starting in 2018 because of problems with the doors and conductors’ emergency brake valves. In addition, the MTA twice halted deliveries of the new cars—in August 2018 and January 2019—when more than eight “units,” or trains, consisting of four or five R179 subway cars, were unavailable for service because of various issues such as problems with doors, heating, ventilation and air conditioning systems, and air compressor units.
Delays Resulting from Welding and Casting Issues
Under the terms of the contract, Bombardier was required to submit to MTA all welding procedures prior to production. Bombardier continually refused to obey the clear contract requirements, citing a desire to protect proprietary information, for nearly two years.
Instead, in violation of the contract, Bombardier proceeded with production of the prototype. In December 2013, Bombardier discovered a problem with welds of the truck frame, deemed “hot cracking.” This critical problem halted production. Yet Bombardier did not report the issue to MTA until February 2014 and in April 2014 finally began submitting the welding procedures they were required to have submitted years earlier.
Ultimately, production was stopped twice due to this issue — first from December 2013 when the cracks were discovered, until June 2014; and then again from September 2014 when the cracks reappeared, until November 2014.  It was not until July 2015, 18 months after the initial discovery of the cracks that a solution to the welding issue was formally approved.
Welding Issue Timeline
DateDescription
August 2012MTA requested Bombardier submit all welding procedures, as required by the TS and Bombardier refused claiming that it would not submit them because of the confidential and proprietary nature of the information.
December 2013Bombardier discovered hot cracking in the truck frame and stopped production.
February 2014Bombardier notified MTA of the hot cracking problem.
March 2014Production of cars was delayed by 50 working days due to Bombardier’s defective welding process.
April 2014Bombardier begins to submit portions of its welding procedures to MTA as required by the Contract.
June 2014New welding procedures were submitted and approved by MTA, production of truck frames resumed with an interim inspection plan in place.
September 2014Hot cracking reappeared in the welds and production was stopped again.
September 2014MTA determined that the defective welding process delayed the project by 13 months.
November 2014Production to resume with an interim inspection plan in place and Bombardier continued to update welding procedures.
June 2015Submission of all welding procedures was completed.
July 2015MTA approves Bombardier’s solution for the Weld Hot Cracking.
In addition, Bombardier failed to adequately oversee the work of its casting supplier and subcontractors, and as a result, received defective castings in need of repair and replacement – causing additional delays.
Delays Resulting from Continuous Failures of Qualification Tests
Bombardier also failed numerous systems qualification tests between early 2014 through mid-2015. Qualification tests are required to be conducted with witnesses from MTA present, unless instructed otherwise. Bombardier did not conduct independent testing prior to the official tests, so defects and failures were first identified when executing the official MTA qualification tests.
These qualification tests are typically one-time tests, yet Bombardier repeatedly failed tests the first time taken and then took over a year to finally pass individual tests.
The below table shows examples of qualification test failures and length of time it took to pass the tests.
Qualification Test Delays
System Test1st Run (Failed)2nd Run (Failed)Test Pass DateNumber of Days to Pass Test
Event Recorder Software Test03/09/201510/21/2015226
Communication and Signs Electromagnetic Interference Test02/10/201405/21/20141/7/2015331
Doors 2 Million Cycle Test10/21/201405/20/2016577
Doors Shock & Vibration Test04/14/201508/18/2015126
Brakes Electric Coupler Shock & Vibration Test06/16/201410/15/2015486
MTA Did Not Fully Exercise Oversight of Contract to Mitigate Delays
The Comptroller’s audit found that Bombardier quickly fell behind in meeting contract milestones. But despite these early and persistent delays, it was not until July 2016 that MTA took some of the actions it was entitled to take to enforce compliance with contract deadlines and other requirements.
Under the contract, MTA is entitled to take certain actions such as requesting adequate assurance and declaring an Event of Default, when a Contractor’s performance is unnecessarily or unreasonably delayed or in violation of contract provisions. MTA representatives told the audit team that they believed Bombardier would catch up to its timeline during production, as had occurred with other vendors, despite this being a relatively small order and their extensive documentation showing that Bombardier was demonstrating an exceptionally subpar performance.
MTA did not conduct annual evaluations of the Bombardier contract as required by its internal guidelines over the course of the seven year contract, instead only performing evaluations in 2016, 2018 and 2019. When MTA conducted an evaluation of Bombardier in 2016, they gave an overall rating of “Marginal” but failed to notify Bombardier until August 2018 – after the Comptroller’s Office inquired.
While delays in meeting contract milestones may certainly occur, and the design and production of subway cars is complicated, by not exercising all available contractual tools and enforcement authority promptly or at all, MTA failed at its oversight authority to mitigate extensive contract delays.
Bombardier Car Delays Resulted in Millions of Dollars in Additional Maintenance Costs
The MTA has not provided a full cost of the delays caused by Bombardier’s failure to deliver R179 subway cars, but initially attributed a $50 million increase in maintenance costs for older subway cars to the delays and increased revenue service fleet requirements. The MTA has now reduced its cost estimates of the new maintenance costs caused by these delays to $35 million.
Recommendations
Comptroller Stringer’s audit included a series of recommendations for MTA to ensure better contract compliance, including:
  • Promptly utilize all enforcement mechanisms afforded under contract authority to mitigate delays early on and issue clear procedures to follow on timing enforcement
  • Ensure that contractors fully understand the specifications in the RFP prior to awarding a contract
  • Conduct annual vendor evaluations and promptly inform vendors of marginal or unsatisfactory ratings and performance deficiencies in writing
  • Clearly document and memorialize high-level communications with vendors
The audit also issued a series of recommendations for Bombardier, including:
  • Strictly adhere to all contract requirements
  • Promptly provide all information to MTA, especially in relation to any failures to meet contract deadlines
  • Manage projects in compliance with approved schedules
  • Institute an effective inspection and oversight program