A Summary of NASA and USAF Hypergolic Propellant Related Spills and Fires
A Summary of NASA and USAF Hypergolic Propellant Related Spills and Fires
A Summary of NASA and USAF Hypergolic Propellant Related Spills and Fires
June 2009
NASA/TP-2009-214769
Since its founding, NASA has been dedicated to CONFERENCE PUBLICATION. Collected
the advancement of aeronautics and space papers from scientific and technical
science. The NASA Scientific and Technical conferences, symposia, seminars, or other
Information (STI) program plays a key part in meetings sponsored or co-sponsored by NASA.
helping NASA maintain this important role.
SPECIAL PUBLICATIONS. Scientific,
The NASA STI program operates under the technical, or historical information from NASA
auspices of the Agency Chief Information Officer. programs, projects, and missions, often
It collects, organizes, provides for archiving, and concerned with subjects having substantial
disseminates NASA’s STI. The NASA STI public interest.
program provides access to the NASA
Aeronautics and Space Database and its public TECHNICAL TRANSLATION. English-
interface, the NASA Technical Report Server, thus language translations of foreign scientific and
providing one of the largest collections of technical material pertinent to NASA’s mission.
aeronautical and space science STI in the world.
Results are published in both non-NASA channels Specialized services also include organizing and
and by NASA in the NASA STI Report Series, publishing research results, distributing specialized
which includes the following report types: research announcements and feeds, providing help
desk and personal search support, and enabling
TECHNICAL PUBLICATION. Reports of data exchange services.
completed research or a major significant
phase of research that present the results of For more information about the NASA STI program,
NASA programs and include extensive data or see the following:
theoretical analysis. Includes compilations of
significant scientific and technical data and Access the NASA STI program home page at
information deemed to be of continuing http://www.sti.nasa.gov
reference value. NASA counterpart of peer-
reviewed formal professional papers, but E-mail your question via the internet to
having less stringent limitations on manuscript [email protected]
length and extent of graphic presentations.
Fax your question to the NASA STI Help Desk
TECHNICAL MEMORANDUM. Scientific and at 443-757-5803
technical findings that are preliminary or of
specialized interest, e.g., quick release
Phone the NASA STI Help Desk at
reports, working papers, and bibliographies
443- 757-5802
that contain minimal annotations. Does not
contain extensive analysis.
Write to:
NASA Center for Aerospace Information (CASI)
CONTRACTOR REPORT. Scientific and
7115 Standard Drive
technical findings by NASA-sponsored
Hanover, MD 21076-1320
contractors and grantees.
NASA/TP-2009-214769
June 2009
NASA/TP-2009-214769
Acknowledgements
The use of trademarks or names of manufacturers in this report is for accurate reporting and does not
constitute an official endorsement, either expressed or implied, of such products or manufacturers by the
National Aeronautics and Space Administration.
Available from:
Page iv
NASA/TP-2009-214769
Abstract
Several unintentional hypergolic fluid related spills, fires, and explosions from the Apollo Program, the
Space Shuttle Program, the Titan Program, and a few others have occurred over the past several
decades. Spill sites include the following government facilities: Kennedy Space Center (KSC), Johnson
Space Center (JSC), White Sands Test Facility (WSTF), Vandenberg Air Force Base (VAFB), Cape
Canaveral Air Force Station (CCAFS), Edwards Air Force Base (EAFB), Little Rock AFB, and McConnell
AFB. Until now, the only method of capturing the lessons learned from these incidents has been “word of
mouth” or by studying each individual incident report.
The root causes and consequences of the incidents vary drastically; however, certain “themes” can be
deduced and utilized for future hypergolic propellant handling. Some of those common “themes” are
summarized below:
Improper configuration control and internal or external human performance shaping factors can
lead to being falsely comfortable with a system
Communication breakdown can escalate an incident to a level where injuries occur and/or
hardware is damaged
Improper propulsion system and ground support system designs can destine a system for failure
Improper training of technicians, engineers, and safety personnel can put lives in danger
Improper PPE, spill protection, and staging of fire extinguishing equipment can result in
unnecessary injuries or hardware damage if an incident occurs
Improper procedural oversight, development, and adherence to the procedure can be detrimental
and quickly lead to an undesirable incident
Improper materials cleanliness or compatibility and chemical reactivity can result in fires or
explosions
Improper established “back-out” and/or emergency safing procedures can escalate an event
The items listed above are only a short list of the issues that should be recognized prior to handling
hypergolic fluids or processing vehicles containing hypergolic propellants. The summary of incidents in
this report is intended to cover many more issues than those listed above.
Page v
NASA/TP-2009-214769
Table of Contents
Abstract ......................................................................................................................................................... v
Table of Contents ......................................................................................................................................... vi
1 Introduction ............................................................................................................................................ 1
1.1 Properties of Nitrogen Tetroxide (N2O4)........................................................................................ 2
1.2 Properties of Hydrazine (N2H4) ..................................................................................................... 3
1.3 Properties of Monomethylhydrazine (MMH) ................................................................................. 5
1.4 Summary of Hypergolic Fluid Properties ...................................................................................... 6
2 Apollo 7 SPS N2O4 Spill (September 1968, CCAFS LC-34).................................................................. 7
3 Apollo-Soyuz Astronaut N2O4 Vapor Exposure (7/24/1975, Apollo-Soyuz Test Project Apollo
Command Module During Re-Entry)............................................................................................................. 9
4 OV-101 APU 1 Cavity Seal N2H4 Spill (6/28/1977, Second Captive-Active Flight) ............................. 11
5 Titan II Silo Large Scale N2O4 Spill (8/24/1978, McConnell AFB Silo 533-7)...................................... 12
6 N2H4 Spill Following APU Hotfire (11/1979, KSC OPF1)..................................................................... 14
7 Titan II Explosion Following A-50 Spill (9/18/1980, Little Rock AFB Silo 374-7)................................. 14
8 KSC Incorrect Flight Cap N2O4 Vapor Release (July 1981, KSC OPF1) ............................................ 17
9 MMH Exposure Following Flexhose Removal at Pad Farm (7/14/1981, KSC Pad 39A Fuel Farm)... 18
10 STS-2 OV-102 Right Pod MMH Fire (Fall 1981, KSC OPF1).......................................................... 18
11 STS-2 OV-102 N2O4 Spill (9/22/1981, KSC Pad 39A 207-Foot Level)............................................ 19
12 Pad 39A Fuel Farm MMH Spill and Fire Following Pneumatic Valve R&R (6/29/1982, KSC Pad
39A Fuel Farm) ........................................................................................................................................... 23
13 N2O4 Vapor Release from Flange Gasket (2/10/1983, KSC Pad 39A Oxidizer Farm) .................... 25
14 FRCS Ferry Plug Removal MMH Spill (4/18/1983, KSC OPF1)...................................................... 25
15 STS-9 OV-102 APU-1 and -2 Explosion (12/8/1983, EAFB Runway 170) ...................................... 26
16 N2O4 Vapor Release from Loose Fitting (2/17/1984, KSC OPF2) ................................................... 28
17 CCAFS Tanker MMH Fire (5/16/1984, CCAFS FSA 1) ................................................................... 29
18 Liquid Trap in Purge Adapter Flexhose MMH Spill (5/24/1985, KSC OPF1)................................... 30
19 STS-61C OV-102 SRB HPU Loading N2H4 Spill (12/8/1985, KSC Pad 39A MLP Surface)............ 31
20 Inadvertent Dry Well Removal MMH Spill (1/21/1986, KSC Pad 39A Fuel Farm)........................... 31
21 Relief Valve R&R Oxidizer Farm N2O4 Vapor Release (7/29/1986, KSC Pad 39A) ........................ 33
22 OPF2 Trench N2H4 Spill and Fire (9/19/1986, KSC OPF2) ............................................................. 34
23 N2O4 and Insulation Adhesive Small Fire (6/23/1988, KSC Pad 39B Oxidizer Farm) ..................... 35
24 STS-26R OV-103 N2O4 Tubing Leak on Vehicle (7/14/1988, KSC Pad 39B) ................................. 36
25 WSTF Fuel Waste Flash Fire (2/16/1990, WSTF) ........................................................................... 40
26 Aspiration of N2O4 into Fuel Vent System (3/26/1990, WSTF TS 401) ........................................... 40
27 HMF Screens Test Drum MMH Spill (12/7/1990, KSC HMF M7-961 East Test Cell) ..................... 41
28 STS-42 OV-103 Ferry Plug Removal MMH Spill (2/12/1992, KSC OPF3)...................................... 41
29 WSTF Incorrect Flight Cap N2O4 Exposure (11/4/1992, WSTF) ..................................................... 42
30 Thermochemical Test Area N2O4 Vapor Release (4/21/1994, JSC Building 353)........................... 42
31 Titan IV A K-9 N2O4 Spill (8/20/1994, CCAFS SLC-41) ................................................................... 43
32 STS-69 OV-105 Left Pod MMH Fire (12/9/1994, KSC OPF1) ......................................................... 46
33 STS-69 OV-105 Right Pod MMH Fire (5/4/1995, KSC OPF1)......................................................... 47
34 ORSU Open Manual Valve N2O4 Spill (3/1/1996, WSTF 400-Area)................................................ 51
35 OPF2 GSE MMH Spill (2/17/1997, KSC OPF2) .............................................................................. 51
36 HMF Sample Valve MMH Spill (3/26/1997, KSC HMF M7-1212 West Test Cell) ........................... 53
37 VAFB Titan IV A K-18 N2O4 Spill (7/16/1997, VAFB SLC-4E) ......................................................... 54
38 Pad 39B Slope N2O4 Spill (11/6/1997, KSC Pad 39B Slope) .......................................................... 58
39 STS-109 OV-102 APU Hydrazine Spill (8/20/1999, KSC OPF3)..................................................... 61
40 WSTF Pathfinder Axial Engine Valve Failure (8/7/2000, WSTF TS 401) ........................................ 62
41 WSTF Pathfinder Small MMH Fire (8/12/2000, WSTF TS 401) ...................................................... 63
42 WSTF Pressure Transducer Explosion (3/25/2003, WSTF TC 831) ............................................... 63
43 Titan IV N2O4 Pump Explosion (8/12/2003, CCAFS LC-40) ............................................................ 65
44 HMF RP01 N2O4 Spill (6/5/2004, KSC HMF M7-961 East Test Cell) .............................................. 68
45 WSTF N2H4 Spill Following Manual Valve Failure (9/30/2005, WSTF TC 844B) ............................ 69
Page vi
NASA/TP-2009-214769
46 STS-121 FRC3 N2O4 Spill (1/9/2006, KSC HMF M7-1212 West Test Cell) .................................... 72
47 Conclusion........................................................................................................................................ 77
48 References ....................................................................................................................................... 81
49 Appendix A: Acronyms and Abbreviations ...................................................................................... 86
50 Appendix B: Summary of Incidents ................................................................................................. 88
51 Appendix C: Detailed Assessment of Incidents .............................................................................. 97
Page vii
NASA/TP-2009-214769
1 Introduction
Hypergolic fluids are toxic liquids that react spontaneously and violently when they contact each other.
These fluids are used in many different rocket and aircraft systems for propulsion and hydraulic power
including, orbiting satellites, manned spacecraft, military aircraft, and deep space probes. Hypergolic
fuels include hydrazine (N2H4) and its derivatives including monomethylhydrazine (MMH), unsymmetrical
di-methylhydrazine (UDMH), and Aerozine 50 (A-50), which is an equal mixture of N2H4 and UDMH. The
oxidizer used with these fuels is usually nitrogen tetroxide (N2O4), also known as dinitrogen tetroxide or
NTO, and various blends of N2O4 with nitric oxide (NO).
Several documented, unintentional hypergolic fluid spills and fires related to the Apollo Program, the
Space Shuttle Program, and several other programs from approximately 1968 through the spring of 2009
have been studied for the primary purpose of extracting the lessons learned. Spill sites include KSC,
JSC, WSTF, CCAFS, EAFB, McConnell AFB, and VAFB. Some spills or fires may not be captured in this
document as a result of it covering several different worksites and spanning several decades.
The Space Transportation System’s (STS) Orbital Maneuvering System and Reaction Control System
(OMS/RCS) use hypergolic propellants to provide on-orbit maneuvering and de-orbit capabilities.
Processing of the Space Shuttle orbiters occurs at the National Aeronautics and Space Administration’s
(NASA) Kennedy Space Center (KSC) near Titusville, Florida; on-orbit operations are managed by
NASA’s Johnson Space Center (JSC) in Houston, Texas; and hypergolic rocket engine checkout and
testing occurs at NASA’s White Sands Test Facility (WSTF) in Las Cruces, New Mexico. The only
exception to processing was when the orbiters were previously sent to Palmdale, California for an Orbiter
Maintenance Down Period (OMDP). For OMDP the OMS/RCS pods/modules were typically removed
from the orbiters (as they are a completely removable and replaceable unit) and transported to KSC’s
Hypergolic Maintenance Facility (HMF).
NASA Procedural Requirement (NPR) 8621.1 revision B was used as a classification guideline to
establish the following mishap related definitions:
NASA Mishap – An unplanned event that results in at least one of the following:
Proximate Cause – The event(s) that occurred, including any condition(s) that existed
immediately before the undesired outcome, directly resulted in its occurrence and, if eliminated or
modified, would have prevented the undesired outcome. The proximate cause is also known as
the direct cause(s).
Root Cause – One of multiple factors (events, conditions, or organizational factors) that
contributed to or created the proximate cause and subsequent undesired outcome and, if
eliminated or modified, would have prevented the undesired outcome. Typically, multiple root
causes contribute to an undesired outcome.
Page 1
NASA/TP-2009-214769
These definitions are utilized for the categorization of all the incidents discussed in this document. A
summary of the categorizations can be seen in Appendix C: Detailed Assessment of Incidents.
The intent of this report is to provide a lessons learned resource that can be utilized for future hypergolic
system designs and operations. There have been several incidents involving injuries and damage to high
value hardware that could have been prevented by utilizing the specific knowledge and preventative
safety measures that are discussed in this document. Numerous engineers were interviewed that have
been involved with the Space Shuttle Program since its genesis and the other programs that are
mentioned. The following incidents capture what they could remember.
The discussions in this report are by no means intended to accuse or “point fingers” by placing blame
where it may not be appropriate. The study of incidents and mishaps is crucial for the successful future of
the space program. Prior to the discussion on the particular incidents, the following three sub-sections
discuss properties of three hypergolic propellants. One must understand the physics and physiological
effects of these chemicals before being able to study and understand incidents involving them.
When N2O4 liquid or NO2 vapors come in contact with skin, eyes, or the respiratory system, the oxides of
nitrogen react with water to produce nitric (HNO3) and nitrous (HONO) acids that typically destroy tissue.
Together, these compounds oxidize the moist and flexible inner tissue of the alveoli sacs within the lungs
when inhaled. The alveoli sacs are the location in which the oxygen and carbon dioxide exchange takes
place that is necessary for respiration. Adequate exposure will cause these affected areas oxidative
stress and cellular death. The pulmonary capillaries are the next to die. When this occurs, the plasma
diffuses through the vessel walls in the lungs, resulting in a build-up of fluid (edema). Since the fluid
accumulation results from pulmonary vessel failure, the effect and symptoms may not be immediate.
However, at high enough concentrations, immediate death from hypoxia could occur as a result of airway
spasm, oxygen displacement, or reflex respiratory arrest. Delayed death could occur as a result of
significant fluid build-up leading to respiratory failure. In non-mortal exposure cases, tissue may heal with
scarring (in the location where the tissue was significantly exposed), leading to bronchiolitis obliterans
(destruction of the small airways and air sacs). Survivors may have varying degrees of permanent
restrictive lung disease with pulmonary fibrosis.
N2O4 (NO2) vapors are approximately three times heavier than air and liquid N2O4 evaporates about five
times faster than water at room temperature. The vapors of MON-3 are usually reddish-brown in color,
which is caused by rapid vaporization of NO2. Liquid N2O4 and its vapors will explode on contact with
hydrazine fuels, amines, and alcohol. Ignition may also occur when N2O4 comes into contact with wood,
paper, hydrocarbon fuels, and some adhesives. A mixture of N2O4 and halogenated solvents: carbon
tetrachloride, TCE, perchloroethylene, etc., may produce a violent explosion. MON-3 N2O4 has the
following properties:
Page 2
NASA/TP-2009-214769
o
Boiling Point (14.7 psia), F 70.1
o
Freezing Point, F 11.8
14.57 Vp[psi] 10^ { 5.247 - 2654 / (T[R]) 0.00175 (T[R]) }
o
Vapor Pressure (70 F), psia
o
Specific Gravity (77 F) 1.423
Ignition Capability Not flammable
Odor Bleach-like
Odor Threshold, ppm 1 to 3
REL, ppm (parts per million) 1.0 (exposure limit for NASA hardware processing)
o
Density (77 F & 14.7 psia), lbm/gal 11.96
Density[lbm/ft3 ] 95.499- 0.07804 (T[ F]) 0.00072 (P[psig])
References: SP-086-2001; Hall; Rathgeber; Myers
Currently, monopropellant grade hydrazine (N2H4) is the fuel used in the Auxiliary Power Units (APU) on
the Space Shuttle orbiters and the Hydraulic Power Units (HPU) on the Space Shuttle Solid Rocket
Boosters (SRBs). N2H4 is also used on many spacecraft for monopropellant rocket propulsion (on the
order of single digits to hundreds of pounds of thrust per rocket engine). To produce thrust,
monopropellant rockets utilize a metal-based agent to catalytically decompose the N2H4 into ammonia,
nitrogen, and hydrogen. Liquid hydrazine contains about 98.5% pure N2H4 with the remaining 1.5% being
primarily water. Aerozine 50 (along with N2O4) was used for the first and second stages of the Titan II
Intercontinental Ballistic Missile (ICBM) and Titan space launch vehicles including the 23G (a variant of
the Titan II used for launching medium-sized spacecraft), IIIB, IIIC, and IV. The Titan II, IIIB, IIIC, and IV
rockets used the largest quantities of hypergols per launch in the history of the United States rocket fleet
(for the first stage approximately 13,000 gallons of N2O4 and 11,000 gallons of A-50 was used along with
3,100 gallons of N2O4 and 1,700 gallons of A-50 for the second stage).
The Occupational Safety and Health Administration (OSHA) classifies N2H4 and its derivatives as a
possible carcinogen. N2H4 and its derivatives are extremely toxic, highly flammable, and highly corrosive.
“Hydrazines and their vapors explode on contact with strong oxidizers, such as N2O4, hydrogen peroxide,
fluorine, and halogen fluorides. Additionally, they react on contact with metallic oxides, such as iron,
copper, lead, manganese, and molybdenum to produce fire or explosion,” as quoted from George Hall.
See Figure 1-1 for more information on hydrazine material compatibility as compared to pure titanium.
Metals to the right of titanium in Figure 1-1 are less compatible with hydrazine than titanium. Metals to
the left are more compatible. Material compatibility is quantified on a relative basis as a result of
variables in the systems that contain the materials including pressures and thermal characteristics.
Hydrazine fires produce little to no smoke or colorful flames. N2H4 has a tendency to react exothermically
with or without an oxidizer present (the reaction increases the temperature thus increasing the reaction
rate; this is also known as a thermal runaway reaction). Another way to describe a hydrazine thermal
runaway reaction is “…the rate of heat generation by the reaction exceeds the rate of heat removal from
the system,” as quoted from F. J. Benz. This process is directly related to the auto-ignition temperature,
which decreases as pressure increases. The exothermic reaction can end in an explosion if one or more
of the following conditions are met within the system containing the hydrazine: the reacting system is
confined to a rigid volume; the reacting system is adiabatic or nearly adiabatic; the reaction rate increases
with temperature; or if the hydrazine is subjected to rapid over-pressurization through “water hammer,” as
stated in SP-085-1999. The flammability regions for MMH and N2H4 are shown in Figure 1-2.
Page 3
NASA/TP-2009-214769
250
Page 4
NASA/TP-2009-214769
The vapor densities of all hydrazines are greater than air. Hydrazine evaporates at approximately the
same rate as water at room temperature. N2H4 liquid at room temperature and pressure is clear and oily.
N2H4 and MMH are hygroscopic (they readily absorbs water); therefore, water is widely used as a diluting
agent. A liquid mixture of 58% water and 42% hydrazine or MMH by weight prevents ignition in an open
air environment. A vapor mixture of 65% water and 35% hydrazine or MMH is considered nonflammable
in air. The following are properties of N2H4:
H H
H H H
C N N
N
H H
N
H
H H
Extra Methyl Group
MMH may have a slight yellow-orange tinted flame. MMH can also react exothermically with or without
an oxidizer present, but the reaction rate has been found to be much slower than N2H4. MMH vapor has
also been found to be much less sensitive to detonation as compared to N2H4. As a result of the
molecular differences in comparison to N2H4, MMH has slightly different properties as shown below:
Page 5
NASA/TP-2009-214769
Figure 1-4: MMH Reactivity Relative to Commercially Pure Titanium (at 353 K)
It may seem odd that with all these seemingly negative characteristics, spacecraft designers still choose
to use hypergols for propulsion systems. This is primarily due to the fact that hypergols are storable and
Page 6
NASA/TP-2009-214769
stable (as long as they are contained properly). They also have a high specific impulse when used for
propulsion, are stable to impact as long as there is no spark, can withstand the extremes of hot and cold
which are present in the vacuum of space, and can be frozen and then thawed without detrimental effects
to the chemical properties or storage vessels since they contract when frozen. However, care needs to
be taken when hypergols are frozen in tubing. The volume of the propellant in the solid state is less than
it is in the liquid state; therefore, as the propellant freezes, additional liquid fills the void created by the
decrease in volume. When the propellant thaws, there is not enough volume to contain the liquid and the
line bursts as a result of an over-pressurization, depending on the thaw pattern in the tubing. This is why
thermal control of tubing is very important in hypergol systems.
Page 7
NASA/TP-2009-214769
Mobile Service
Structure
Launch Pad
Figure 2-2: LC-34 Launch Pad and Service Structure (In Park Position)
The Apollo SPS hypergol loading ground support equipment (GSE) consisted of two six-wheeled, flat bed
portable tanker trailers (one for oxidizer and one for fuel) that held heavy-walled storage tanks, control
valves, and a control console. Prior to the loading operation, the trailer was staged approximately 200
feet from the base of the pad structure. The servicing GSE also included the portable servicing panels
that were staged at the base of the LC-34 structure. These large panels (15 feet long, by 10 feet high, by
8 feet wide) contained pumps, valves, and electrical cabinets inside an enclosure. Portable air dilution
scrubbers with liquid separators were also located at the base of the launch pad separate to the trailers.
Each had three big fans to disperse the propellant vapor. Hard-line tubing carried the propellant up the
launch pad structure. Flexhoses were connected to the hard-lines by flanges to transfer the propellant
from the fixed structure to the mobile structure propellant delivery system, which consisted of a valve box
with supply and return lines and a crossover three-way valve.
The N2O4 spill occurred when a technician was disconnecting the two-inch hard-line/flexhose flange.
Engineers thought that they had performed an adequate drain and purge of the line; however, the
technician discovered that this was not the case (proximate cause). When he began to unbolt the flange,
about one to two gallons of liquid N2O4 poured out of the line onto the mobile Service Structure and flight
vehicle. Liquid N2O4 ran down the side of the vehicle and into the instrument unit ring, as seen in Figure
2-3 in the Saturn 1B cutaway view. The liquid N2O4 was diluted with water, which turned out to be a poor
decision since N2O4 and water form nitric and nitrous acids. A de-stack was completed to remove the
instrument unit ring for repair.
It was found that the GSE propellant tubing contained a low point that trapped the liquid, not allowing it to
be purged through the system during the post-operations of the loading procedure (root cause). This
incident led to the requirement to install “spill protection” and “scuppers” onto the vehicle to capture any
spilled liquid from the vehicle. The scupper is a box that is attached to the vehicle servicing door that
serves the dual purpose of spill containment and carrying the loads that the external GSE hoses place on
the vehicle servicing valves and fluid lines. An additional requirement was also developed mandating that
GSE designs be free of low point liquid traps.
Page 8
NASA/TP-2009-214769
During reentry, the crew inadvertently performed a few tasks out sequence (root cause). The first task in
the series was the opening of the cabin pressure relief valve to gradually equalize the cabin pressure with
the outside air as the external ambient pressure increased during the capsule’s descent. The following
task was the manual deployment of the drogue parachute at approximately 23,000 feet. This activity
caused the capsule to sway. The onboard guidance, navigation, and control computers sensed this
motion and activated the Apollo command module’s reaction control system (RCS) thrusters to counteract
it (proximate cause). Non-combusted N2O4 (NO2) vapors were subsequently drawn into the capsule
through the open cabin relief valve exit port about two feet from the closest RCS thruster. About 30
seconds later, the crew deactivated the RCS thrusters; however, the capsule had already filled with N2O4
(NO2) vapors. The thruster deactivation had been planned to be completed prior to the opening of the
cabin pressure relief valve.
Page 9
NASA/TP-2009-214769
Brand describes the reentry events during the crew technical debriefing:
“At 30K [30,000 feet], normally we arm the ELS [Earth Landing System] AUTO, ELS LOGIC, that
didn’t get done. Probably due to a combination of circumstance, I didn’t hear it called out, maybe
it wasn’t called out. Any case 30K to 24K we passed through that regime very quickly. I looked
at the altimeter at 24K, and didn’t see the expected apex cover come off. Didn’t see the drogues
come out. So, I think at about 23K, I hit the two manual switches. One for the apex cover and
also, the one for drogues. They came out. That same instant the cabin seemed to flood with a
noxious gas, very high concentration it seemed to us. Tom said he could see it. I don’t
remember for sure now, if I was seeing it, but I certainly knew it was there. I was feeling it and
smelling it. It irritated the skin a little bit, and the eyes a little bit, and, of course, you could smell
it. We started coughing. About that time, we armed the automatic system, the ELS…”
The exposure greatly altered the astronauts’ abilities to complete assigned tasks in a timely manner;
therefore, they experienced a very rough landing along with the poor luck of having the capsule splash-
down in the “Stable 2” configuration, which meant it was inverted in the water. Following splash-down,
the crew was forced to don their oxygen masks, but in the interim before Stafford could retrieve and
distribute the masks, Brand (who was sitting closest to the relief valve, see Figure 3-2) lost
coconsciousness.
“For some reason, I was more tolerant to [the NO2 vapors], and I just thought get those damn
masks. I said don’t fall down into the tunnel. I came loose and…had to crawl…and bend over to
get the masks…l knew that I had a toxic hypoxia…and I started to grunt-breathe to make sure I
got pressure in my lungs to keep my head clear. I looked over at Vance [Brand] and he was just
hanging in his straps. He was unconscious.”
Stafford climbed around the module to Brand and placed an oxygen mask over his mouth. With the
oxygen mask in place, Brand regained consciousness within approximately one minute. The capsule was
then reoriented upright and Stafford opened the vent valve, dissipating the remaining vapors. All three
astronauts subsequently developed pneumonitis, for which they were hospitalized for about two weeks in
Honolulu, Hawaii. All three recovered completely. Deke Slayton and Thomas Stafford did not fly on any
subsequent space missions. Vance Brand went on to be the commander for three Space Shuttle flights:
STS-5, STS-41B, and STS-35.
Page 10
NASA/TP-2009-214769
4 OV-101 APU 1 Cavity Seal N2H4 Spill (6/28/1977, Second Captive-Active Flight)
On June 28, 1977, during Enterprise’s (OV-101) second captive-active flight with the crew of Joe Engle
and Richard Truly, approximately five gallons of hydrazine leaked from APU number one’s cavity shaft
seals and dumped overboard via the drain vent at the aft end of the vehicle. There was not a catch bottle
in this early design. The incident resulted in a new design of the shaft seals and the addition of a catch
bottle.
The aerodynamic slipstream of the vehicle caused the hydrazine to be ingested into the left hand side of
the aft fuselage through an access panel and vent door. There were no reported fires. There was
extensive damage to the polyimide “Kapton” insulated wiring and interior thermal blankets near the left
hand APU service panel from hydrazine exposure. There was also damage to the exterior thermal
blankets. This incident seemed to bring to light the incompatibility between hydrazine and Kapton. For
future flights, the left- and right-hand vent ports were modified to eliminate the possibility of ingestion.
Also, the left and right access doors and T-0 umbilical doors were sealed. The root cause of the leak was
an inadequate understanding of the flight characteristics of the APU system. Finding these types of
design faults are, however, the exact reason that flight tests are performed.
Page 11
NASA/TP-2009-214769
5 Titan II Silo Large Scale N2O4 Spill (8/24/1978, McConnell AFB Silo 533-7)
On August 24, 1978, at McConnell AFB southeast of Wichita, Kansas, the worst reported unintentional
N2O4 spill in U.S. history took place. Approximately 13,450 gallons of liquid N2O4 spilled into the
st
underground missile silo. Two members of the 381 Strategic Missile Wing were killed and 25 more were
injured from N2O4 liquid and vapor exposure. The silo also suffered extensive damage. The Titan II first
and second stages were an A-50/N2O4 bi-propellant propulsion system. The loading operations were
completed using a holding tanker located above ground. Recall that N2O4 (NO2) vapors are heavier than
o
air and they boil at approximately 70 F at ambient pressure.
Once the N2O4 loading of the rocket’s first and second stages had been completed, all that remained was
disconnecting the quick disconnects (QDs) from the rocket air half couplings (AHC). The technicians
wearing Rocket Fuel Handler Coverall Outfits (RFHCOs) were unaware that a Teflon o-ring seal had
dislodged from an upstream location, moved through the filter assembly (of which there were no filter
elements installed), through the QD, and wedged itself in between the poppet of the AHC and its primary
sealing surface on the Titan II (proximate cause) of the spill. This prevented the isolation of the bulk
propellant in the first stage oxidizer tanks once the QD was removed. The RFHCOs were similar to Self-
Contained Atmospheric Protection Ensemble (SCAPE) suits that are used by the Space Shuttle Program.
When the technician mechanically separated the QD from the AHC, approximately 13,450 gallons of
liquid N2O4 poured out of the AHC and into the missile silo.
Very high concentrations of NO2 vapors traveled from the silo into the connecting cableway to the blast
lock area, which was located just outside of the control center where several personnel were located (see
Figure 5-1). All the personnel located in the blast lock area managed to escape the vapors by exiting
through the access portal at ground level. The personnel in the control center would normally be isolated
from the silo; however, two technicians involved in the spill opened the airlock door to try and obtain
assistance for their supervisor who was in need of immediate medical attention. The two technicians and
four other control center personnel then evacuated through the emergency escape hatch, once they
realized the immediate danger of the situation. The nearby town of Rock, Kansas had to evacuate about
100 people as a result of the vapors that were escaping from the silo (see Figure 5-2).
It was later discovered that the suit of the supervisor had failed, introducing him to dangerous levels of
N2O4 (NO2) vapors, which was ultimately fatal to him within minutes of being exposed (see section 1.1
“Properties of Nitrogen Tetroxide (N2O4)” for a description of what N2O4 and NO2 does to the body). Two
other technicians had removed their suit hoods while in the vapor cloud. One died nine days later. The
extent of the injuries of the other technician is uncertain; however, he did survive. Repairs to the
damaged 533-7 silo were attempted, but later discontinued for budgetary reasons. The silo was never
returned to alert status.
Proper configuration control of GSE components, in this case the filter, is highly important in the
handling of toxic chemicals, especially hypergols (root cause of the incident)
Emergency procedures and safing must always be reviewed, practiced, and ingrained in the
minds of personnel working with toxic chemicals
Procedural oversight may have been beneficial
Proper isolation of bulk propellant should be inherently designed into any rocket propulsion
system and used accordingly during loading operations
Personal protective equipment (PPE) should be inspected prior to every use
Page 12
NASA/TP-2009-214769
Emergency
Escape Hatch
Blast Lock Area
Control Center
Figure 5-2: Arial View of N2O4 Vapor Cloud Coming from Missile Silo 533-7
Page 13
NASA/TP-2009-214769
7 Titan II Explosion Following A-50 Spill (9/18/1980, Little Rock AFB Silo 374-7)
On September 19, 1980, following a large A-50 spill the previous evening, the Titan II ICBM within missile
th
silo 374-7 located about 2.5 miles south of Bee Branch, Arkansas, exploded. One of the 308 Strategic
Missile Wing’s airmen was killed and 21 other USAF personnel were injured as a result of the explosion
and subsequent rescue operations.
At about 6:30 PM on September 18, 1980, an airman was conducting a maintenance operation on level
two (see Figure 7-1) of the underground silo when he accidentally dropped a large wrench socket. The
standing platforms were hydraulically-controlled, flip-down structures with a rubber boot mounted
between the platform and the rocket. The socket fell hitting the standing level two platform and bounced
in the direction of the rocket where it slipped through the small gap between the rubber boot (which had
become pliable over the years) and the Titan II rocket. It fell about 70 feet before hitting the thrust mount
near the base of the rocket. The socket then bounced into and ruptured the stage one fuel tank
(proximate cause). Approximately 11,140 gallons of A-50 drained into the bottom of the silo. Fuel vapors
heated up the silo and caused the pressures in the non-ruptured propellant tanks to rise substantially.
At about 8:00 PM the control center was evacuated (see Figure 5-1 for a view of the launch silo and
supporting facilities) and, therefore, the capability to remotely monitor the silo and rocket system data was
lost. The entire missile complex and the surrounding area were then evacuated and a team of specialists
that were knowledgeable of the Titan II rocket system were called in from Little Rock AFB (the missile’s
main support base). Also, at around this time, local residents within a one-mile radius of the missile silo
were evacuated. Local law enforcement officers closed the nearby State Highway 65, and alerted
anyone entering the area.
At 3:00 AM on the following morning (September 19, 1980) two people entered the control center in
protective suits through the emergency escape hatch. They were forced to leave shortly thereafter as a
result of the high fuel vapor concentration causing poor visibility. Prior to leaving, one of the men
reportedly activated the exhaust fans which pulled the fuel vapors into an equipment area where some
electrical pumps were located. It is assumed that this is where the fire originated, however this was never
completely proven. It is unclear if the men received orders from a superior officer to activate the exhaust
fans or not. Following the exhaust fan activation, the two men went back to the surface and had just
paused to await further instructions when the Titan II rocket exploded, sending an earthquake-like
shockwave across north central Arkansas. The heat from the flames at the base of the silo increased the
temperature at the lower end of the rocket. This eventually led to the rupture of the N2O4 tank, which
reacted hypergolically with the spilled fuel causing the explosion. One of the two men died later that day
in the hospital (once he was located among the rubble from the explosion). The other man was thrown
approximately 150 feet from the silo and suffered only a broken leg along with several cuts and bruises.
Page 14
NASA/TP-2009-214769
It was reported that the explosion blew the 740-ton reinforced concrete and steel silo door (see Figure 7-2
for a depiction of the Titan II missile silo) 200 feet into the air and 600 feet from the silo. The rocket’s
second stage, with the W53 thermonuclear warhead attached, was launched out of the silo following the
explosion of the first stage. The second stage then supposedly blew up in midair, (it contained about
1,730 gallons of A-50 and 3,120 gallons of N2O4) sending the undetonated warhead several hundred feet
from the silo. The W53 warhead had a mass of 8,136 lb and a yield of 9,000 kilotons (the Hiroshima
bomb “Little Boy” was estimated at about 15 kilotons). Luckily the warhead’s safety features operated as
they were designed. There was no reported loss of radioactive material.
The 374-7 Titan II missile silo complex was completely destroyed. The estimated value of the silo in 1980
was approximately $225,000,000. In October of 1981, President Reagan announced that all of the Titan
II ICBM launch sites across the United States would be deactivated by October of 1987. Along with this
action being part of the strategic modernization program, the deactivation was related to this incident and
the previously mentioned incident at McConnell AFB. Silo 374-7 was the first Titan II silo to be
th
deactivated. The 308 strategic missile wing was completely deactivated on August 18, 1987.
Some lessons learned and corrective actions from this incident include:
Page 15
NASA/TP-2009-214769
Page 16
NASA/TP-2009-214769
8 KSC Incorrect Flight Cap N2O4 Vapor Release (July 1981, KSC OPF1)
On roughly July 14, 1981 in OPF1, there was an inadvertent release of N2O4 (NO2) vapors during the
MD338 flight cap installation procedure. The cap was removed from MD338 on June 15, 1981 and
routed to another facility with a fume hood for refurbishment. During the refurbishment procedure the
incorrect part number (MC276-0018-2411, a ¼-inch flight cap, see Figure 8-1 for a description of the
numbering nomenclature) was recorded for the cap and a parts tag with the incorrect number was
attached. The correct part number was MC276-0018-2811 (a ½-inch oxidizer flight cap, shown in Figure
8-2). There had been issues with oxidizer flight caps becoming corroded; therefore, a problem report
(PR) was generated to clean the caps. During the cleaning procedure the part number that was etched
onto the cap was sanded off and all that was left to identify the cap was the attached parts tag (which was
incorrect).
Coupling Size Poppet Area Force to Open Force to Open Poppet Travel
2
(in.) (in. ) (lbs) (psid) (in.)
¼ 0.709 9.50 13.40 0.034
½ 1.131 14.00 12.38 0.053
1.00 2.193 14.80 6.75 0.139
MD276-0018-XXXX
When this is a “4” the AHC tube size is ¼-inch OD
When this is an “8” the AHC tube size is ½-inch OD
When this is a “6” the AHC tube size is 1.00-inch OD
During the MD338 flight cap installation procedure, on roughly July 14, 1981, the work step correctly
called for the installation of a ½-inch cap (part number MC276-0018-2811) onto the AHC. The cap that
was staged for installation was the misidentified ¼-inch cap. When a ¼-inch cap is placed onto a ½-inch
Page 17
NASA/TP-2009-214769
AHC, the dimensions of the stem allow for the actuation of the ½-inch AHC poppet (reference Figure 29-1
for an illustration of this) if the proper force is applied to the cap and poppet. This is exactly what
happened. An oxidizer vapor release was reported when the technician tried to install the ¼-inch cap
(proximate cause). The root causes are human error, improper configuration management, and improper
vehicle design. The original flight cap design should not have enabled this type of incident to occur. The
technician received minor injuries from exposure to NO2 vapors. Close attention needs to be paid to the
identification of flight caps and other hardware to prevent an event like this from occurring again (which it
does on November 4, 1992 at WSTF).
9 MMH Exposure Following Flexhose Removal at Pad Farm (7/14/1981, KSC Pad
39A Fuel Farm)
On the morning of July 14, 1981, a technician was exposed to MMH liquid and vapors while removing an
improperly labeled GSE flexhose from a panel at the Pad 39A (Space Shuttle and Apollo Program launch
pad, along with 39B) fuel farm (proximate cause). The liquid MMH sprayed onto the technician’s arm and
face, which he immediately attempted to wash off with water. The technician then reported to his
supervisor who instructed him to wash his arm and face again, thoroughly, and formally report the
incident.
Upon investigation, it was found that the flexhose that was being removed was not labeled as hazardous.
Proper labeling and configuration management guidelines were not followed (a root cause). The
procedure had also not received a proper review by an experienced engineer (another root cause). The
engineer that had been assigned to the procedure development task had delegated it to an engineer on
loan from another facility and had instructed him to process the procedure as non-hazardous. Currently,
procedures at KSC require a second review by a qualified engineer for hazardous GSE and vehicle
operations. The technician was very lucky the carelessness in the procedure development and flexhose
labels did not result in a more severe injury.
10 STS-2 OV-102 Right Pod MMH Fire (Fall 1981, KSC OPF1)
In the fall of 1981, a small amount of MMH (approximately a teaspoon) was spilled onto the gold multi-
layer insulation (MLI) blankets in the right OMS pod (RV01) of Columbia (OV-102). Technicians
unknowingly opened a line that contained a small amount of liquid MMH (proximate cause of the spill).
Apparently, instructions had been given to the technicians to remove the blankets and install spill
protection, but this was not completed and it was not incorporated into the procedure. The gold foil of the
MLI blanket acted as a catalyst while the blanket batting absorbed and concentrated the released MMH.
The cause of the fire was a result of the ventilated surface area (by aspirator) creating the correct
conditions for combustion of MMH (proximate cause of the fire). The batting acted as an insulator
effectively containing the heat of the reaction, transferring the heat back to the MMH, and allowing the
o
temperature to increase to the boiling temperature of MMH (189.5 F). Ignition of the blanket followed
o
once the vapor fumes reached the auto-ignition temperature of MMH (382 F). A technician used nearby
flame retardant coveralls to extinguish the flames.
It was later determined that MMH and gold are not compatible. The two root causes of this incident were
operational human error and improper design (incompatible materials). Silver was later used, rather than
gold, for thermal blanket construction. Note: in the past on other programs, the catalyst beds for mono-
propellant thrusters contained gold until the material was switched to platinum. The gold was used for its
high reactivity with hydrazine to support combustion in the thrusters.
Page 18
NASA/TP-2009-214769
11 STS-2 OV-102 N2O4 Spill (9/22/1981, KSC Pad 39A 207-Foot Level)
At 1:13 AM on September 22, 1981 an N2O4 spill occurred at Pad 39A. The proximate cause of the spill
was a failed ground half coupling (GHC), MD162, at the AP28-12 door on FRC2 of OV-102 (Columbia).
Failure of the GHC was a result of iron nitrate build-up between the probe and the dynamic head of the
GHC along with the tight tolerances of the GHC. See Figure 11-1, Figure 11-2, and Figure 11-3 for a
description of the AHC and GHC flow paths and also the failure location. Between 15 to 20 gallons of
oxidizer was released into the attached scupper, which subsequently overflowed onto the vehicle.
Damage to thermal tile adhesive resulted in the removal of 370 tiles (shown in orange in Figure 11-4).
The photograph in Figure 11-5 shows the thermal tiles and the underlying vehicle structure along with the
scupper and QD flexhoses going into the AP28-00 door (second FRCS oxidizer servicing door just
beneath the AP28-12 door when the vehicle is in the vertical orientation).
Confusion resulted in wasted time. Immediately following the spill, the proper alarms went off at the
launch pad’s 207’ level; however, in the Launch Control Center (LCC), the engineers on console were
unaware of the spill source until there was visual confirmation from the technicians (who were located
outside the FRCS room). Engineers on console executed the prewritten worksteps in the Operations and
Maintenance Instruction (OMI) to safe the system once they realized the situation. Unfortunately, these
steps did not include isolation of the QD from the GSE supply; therefore, the leak continued until this was
noted and resolved.
The spill protection was not suitable for a large leak. It was incorrectly assumed that if a spill did occur; it
would be small and containable within the spill protection. The spill protection still in use on the Space
Shuttle Program over 27 years later is very similar to the “temporary” redesigned spill protection following
the STS-2 N2O4 spill.
The launch was delayed by about one month while repairs were made to the vehicle, which remained at
the launch pad. Many of the thermal tiles were baked in an oven and reinstalled onto the orbiter, which is
a standard procedure to remove the N2O4 or NO2 from thermal tiles when they become impregnated with
oxidizer vapors. Several damaged thermal blankets located inside the forward module were replaced at
the pad by accessing them through two doors adjacent to the orbiter windows.
A committee was formed to investigate the spill and compile recommendations for improvements and
lessons learned. The following is a summary of the lessons learned:
The GHC design was flawed in that there was a single point failure resulting in a leak path (a root
cause)
The scupper and apron were not large enough to contain the spill
Using the GHC as a shut-off valve was flawed engineering practice (another root cause)
The emergency procedure was inadequate
All entry paths to the FRCS module should be sealed
Care must be taken in the control of iron nitrate which is always present in an N2O4 system
o Iron nitrate and its impacts to hardware were not well understood in 1981
Proper ventilation and lighting should be added to the FRCS room on the 207-foot level of launch
pads 39A and 39B
Emergency Launch Processing System (LPS) programs could have saved time during safing
The communications system could have been used more efficiently
Corroded structural components that had been exposed to oxidizer vapors were later found in the internal
portion of the FRCS module. It is believed that the vapors entered the forward module through small test
ports located at the external doors. Many years later it was also found that the oxidizer vapor reacted
with several electrical connector backshells within the FRCS.
Page 19
NASA/TP-2009-214769
Failure Location
Figure 11-3: AHC Closed and GHC Failed Open with External Leak Path Shown
Page 20
NASA/TP-2009-214769
Orbiter
Nose
AP28-12 Door
AP28-00 Door
Orbiter
Windows
Figure 11-4: Removed Tile from OV-102 Following STS-2 N2O4 Spill
Page 21
NASA/TP-2009-214769
Figure 11-5: Photograph of AP28-00 Door and Orbiter Following the Spill
A large amount of knowledge was gained from the spill. There were many process and GSE design
changes implemented following the recommendations from the Mishap Investigation Committee Report.
The following is a list of what was completed:
Additional GSE valves were added to isolate the liquid N2O4 rather than using the QDs as valves
o GHCs were subsequently no longer used as shut-off valves during loading
The scuppers were upgraded through a redesign
The QDs and AHCs were found to have very tight tolerances; therefore, the poppets were
subsequently electropolished to open the tolerances
An improved maintenance plan was implemented for the GHCs
Improved local emergency procedures were implemented
Entry paths to the FRCS module were blocked with tape and RTV (adhesive)
Improved controls were put in place to minimize the amount of iron nitrate in the liquid N2O4
The lighting in the FRCS room was enhanced
LPS improvements were made including automatic remote safing that was keyed from local toxic
vapor detection devices
Page 22
NASA/TP-2009-214769
12 Pad 39A Fuel Farm MMH Spill and Fire Following Pneumatic Valve R&R
(6/29/1982, KSC Pad 39A Fuel Farm)
o
On June 29, 1982 (a hot summer day with a high of 90 F), at the Pad 39A fuel farm, there was an MMH
spill and fire during the removal and replacement of a pneumatically controlled valve. Figure 12-1 and
Figure 12-2 are photographs taken following the fire. Prior to the removal and replacement of the valve,
the farm was powered down and the GN2 valve control pressure was removed to enable the removal of
the control pressure tubing on the pneumatic valve. When this pressure was removed, a few valves that
were intended to remain closed, went to open (the valves were normally open valves with the pressure
removed). This was the proximate cause. The change in valve positions went unnoticed or was ignored
by the engineers on console.
Immediately following the removal of the valve a small amount of fuel vapor was released from the open
line. About a minute later, a 12- to 48-inch geyser of MMH was released from the open line and splashed
onto a metal cable tray above and ignited either as a result of the hot metal or by some local iron oxide
(rust). It was estimated that approximately 15 to 25 gallons spilled from the line. The engineer located in
the LCC could not see the events as they occurred as a result of having a view from the Pad 39A oxidizer
farm camera on his closed circuit television (OTV) screen, which is located approximately 1,800 feet from
the fuel farm.
The technicians reported the spill and immediately evacuated the farm to the camera embankment south
of the fuel farm, removing themselves from the communications loop in the process. Once there, they
awaited the SCAPE pickup van. The farm firex was then activated remotely by the duty officer,
extinguishing the fire. The technicians reported that they were unable to reach the firex controls as a
result of the flames. No one was injured in the spill and fire; however, there was a notable amount of
damage to the GSE at the fuel farm as seen in Figure 12-1 and Figure 12-2.
Following the fire, manual overrides were added to the liquid return isolation valve and the storage tank
isolation valve, which cycled normally open when control pressure was removed. Improper GSE design
was one root cause of this incident. It was also found that when the 750 psig GN2 supply pressure was
vented in preparation for the valve removal, the toxic vapor aspirator lost its pneumatic supply pressure.
A pneumatic actuation valve and vent valve were added to the GSE at the propellant farms to better
isolate the liquid MMH and the aspirator, respectively, when removing and replacing valves. Labels were
painted in large letters on the farm roofs and sides of the propellant storage tanks to aid in the
identification of the farms via OTV. Other findings related to this incident include the following:
The procedure was not written or reviewed by an experienced engineer prior to the task
The engineer on console was making changes to the procedure real-time while monitoring
another task in parallel
o Improper adherence to the procedure and the procedure approval process are also root
causes of this incident
There was only one engineer on console supporting the hazardous operation
o The engineer had to leave the communications channel to talk to the test conductor and
in doing this missed some reports from the personnel at the fuel farm
There was not a charged water line nearby when the fire occurred
Page 23
NASA/TP-2009-214769
Separate Pneumatic
Valve Installed in System
Page 24
NASA/TP-2009-214769
13 N2O4 Vapor Release from Flange Gasket (2/10/1983, KSC Pad 39A Oxidizer
Farm)
During processing for the first flight (STS-6) of Challenger on February 10, 1983, there was an N2O4
vapor release at the Pad 39A oxidizer farm. Technicians and engineers were working a decay and leak
check procedure on the fluid distribution system. The engineer on console was bringing up the system
pressure at the farm. While this was occurring, the technicians reported a large N2O4 vapor release.
They also later described hearing a loud noise during the pressurization. The engineer secured the GSE
and reported that there was an emergency on the communications net. The oxidizer farm was cleared of
all personnel and a SCAPE crew was sent in. The SCAPE technicians noted that a flange gasket on an
isolation valve had blown out and was releasing N2O4 (NO2) vapors (proximate cause). There were no
injuries since all personnel at the farm were located upwind from the vapor release. This incident seems
to have been properly managed by the technicians and engineers; however, there may have been a
design flaw in the GSE, which could be considered a root cause.
Reference: Kamp
The liquid presence in the thruster chambers was likely a result of the fuel pilot operated valves leaking.
o
These valves are sensitive to temperatures below 60 F because the Teflon seals non-uniformly contract.
o
Normally, the temperature was maintained above 60 F using the thruster heaters; however, when the
SCA and orbiter landed at KSC, the orbiter was not powered for about 36 hours. During this time period,
o o
the outside temperature dropped to a low of approximately 50 F and remained below 60 F for about
nine hours. The following are lessons learned and corrective actions that were implemented following
this incident:
Thruster heaters shall remain on during all ferry and post-landing orbiter operations
Ferry plug removal was upgraded from the current PPE level at the time to a SCAPE operation
The ferry plug relief valve shall be aspirated with the fuel aspirator prior to ferry plug removal
o If there is any indication of oxidizer at the relief valve exit, the oxidizer aspirator shall be
used for this operation
The proximate cause of this incident was removing a thruster ferry plug without knowing that there was
liquid fuel present behind it. The root cause was an improper operational understanding of the limitations
and sensitivities of the thruster fuel valves.
It was also noted in the corrective actions memo written by Mr. Tribe that it is less likely that the oxidizer
thruster valves would leak since the Teflon seals swell by approximately 3% when exposed to N2O4 at
ambient temperature. The seals only swell by approximately 0.7% in the presence of MMH at ambient
temperature.
Page 25
NASA/TP-2009-214769
Failure analysis indicated that stress corrosion cracking of the injector tube created the leak into APU-1’s
gas generator cavity (proximate cause). In the vacuum of space, the hydrazine froze in the cavity and
remained stable throughout the orbital phase of the mission. However, when the vehicle started its re-
entry, the hydrazine thawed, expanded, vaporized, and then eventually caused the gas generator to
explode. The following is quoted from the Space Shuttle Mission Evaluation Room (MER) database:
“Post-flight data review indicated that hydrazine leakage first occurred approximately 17 minutes
after APU-1 and -2 were started for entry. This condition was indicated by valve-module cooling
caused by hydrazine evaporation. The hydrazine accumulated in an ice state between the valve-
mounting plates and the gas-generator radiation shield. As entry continued and the lower
altitudes were reached, flash evaporation ceased, melting began, and the liquid hydrazine ran
down on to the hot turbine housing surfaces. The ambient pressure in the aft fuselage reached a
level that would support decomposition at approximately 4 minutes and 30 seconds prior to
landing. Hydrazine decomposition and subsequent release occurred as indicated by valve
module heating approximately 4 minutes prior to landing for APU-1 and 2 minutes prior to landing
for APU-2. Numerous instrumentation and electrical wires on both APUs were damaged by fire.
The APU-1 shutoff valve electrical current was interrupted, closing the modulation valve which
caused an APU underspeed condition. The system fuel isolation valve also closed, automatically
isolating the APU-1 fuel supply. Residual heat from the fire, combined with normal heat soak-
o
back, caused the modulation valve and associated tubing to overheat to approximately 500 F.
The trapped hydrazine explosively decomposed and the APU-1 modulation valve detonated. The
detonation caused the APU-1 high-point bleed quick-disconnect poppet to be expelled through
the flight cap and sent shock waves up the fuel line which detonated fuel vapor bubbles in the fuel
pump cavity. Additional hydrazine was sprayed into the aft compartment at the time of APU-1
detonation as indicated by the splash pattern on the avionics bays. Apparently, the shockwave
from the APU-1 detonation caused the already damaged wires on APU-2 to short, closing the
modulation valve causing APU-2 to shut down. This resulted in an automatic isolation valve
closure which isolated the APU-2 fuel supply. The residual heat from the fire combined with the
normal heat soak-back to cause the APU-2 modulation valve to detonate resulting in a high-point
bleed quick disconnect blow off and a subsequent fuel pump detonation.
Inspection of the aft compartment at the APU-1 location revealed minor hydrazine splash in the
area of the APU mounts and on top of the avionics bays. There was smoke and heat
discolorations on the insulation and structure forward of the APU and on the exhaust duct above
the APU. Minor shrapnel damage was noted.
APU-2 had a splash pattern similar to APU-1, except more extensive. The smoke and heat
discolorations were evident to a greater degree than on APU-1 and at locations higher above the
APU. Also, minor shrapnel damage was noted.
The tear-down and inspection of both APUs revealed that the damage was similar and limited to
the fuel systems and wiring. Further inspection of the APU injector tubes revealed that both
tubes were cracked circumferentially upstream of the thermal shunt.
Page 26
NASA/TP-2009-214769
The APU stems had intergranular cracks from the inside diameter to the outside diameter for 225
degrees on APU-1 and 180 degrees on APU-2 around the circumference of the stems. All
microstructure indicated intergranular carbide precipitation at the inside diameter. The most
probable scenario describing the cause of the APU stem failure is as follows:
During the manufacturing operations (braze cycle), a slow cooling of Hasteloy B, which is the
o o
APU stem injector material, from 2100 F to 1100 F resulted in carbide precipitation at the
material grain boundaries. Additional carbon believed to be available from electro discharge
machining of the stem bore diffused into the alloy during the brazing operations. A variance in
cooling rate between the inside diameter and outside diameter during the braze cycle caused
enhanced carbide precipitation near the inside diameter. The resultant microstructure was
sensitized, which means that the corrosion resistance of the grain boundaries was reduced. The
sensitized surface contacted an aggressive environment (hydrazine, air, moisture, carbon
dioxide, ammonia) with attack accelerated at a region-of-stress concentration due to sustained
stress levels (injector stem preload caused by manufacturing assembly misalignment). The crack
progressed until stress levels and/or availability of corrodant changes allowed the fracture to
finish under mechanical or thermal fatigue conditions. The most suspicious corrodant is carbazic
acid. The above scenario is considered to be time dependent. The failure mechanism is thought
to be stress corrosion which requires a susceptible material, an available corrodant, and the
presence of a sustained surface tensile stress level.
The APU failures most probably resulted from a crack in the injector stem caused by corrosion of
the sensitized inside diameter surface. The corrodant is probably carbazic acid or some similar
substance which can be derived from air, moisture, CO2, hydrazine and/or ammonia. The
corrosion is time dependent and the crack progressed under sustained stress levels from the
inner diameter surface toward the outer diameter surface until mechanical or thermal fatigue
conditions could [complete] the crack rupture.”
Page 27
NASA/TP-2009-214769
APU-2
APU-1
Following the STS-9 flight, several changes were made to the APU subsystem, including soaking of the
carbon seals in hydrazine prior to installation, chromizing the injector stems to prevent corrosion, and
minimizing stresses on the injector stems during manufacturing and installation. These design flaws are
believed to be the root causes of the incident.
The actual incident was not that noteworthy, but there were several improvements made to the GSE,
configuration control and management, contractor oversight procedures, and OPF area warning system.
It was found that there was not a proper system in place for a single bay evacuation (OPF1 and OPF2 are
connected). The single bay clears to date had all been conducted through the area paging system, which
most personnel in the processing bay either cannot hear or tune out as a result of the large quantity of
pages that do not concern them. It was also discovered that once the bay had been cleared that it was
impossible to know if everyone was out of the building. These two items resulted in an OPF area warning
system with audible alarms (also known as “warblers”) and flashing lights along with the addition of a
badge board outside the entrance to the bay where personnel are required to place their KSC badge prior
to entering. This was a substantial improvement in safety. With the new area warning system, personnel
were able to be notified in the case of a single bay evacuation or dual-bay evacuation. Also, following this
Page 28
NASA/TP-2009-214769
incident, there was a strong emphasis placed on detailed contractor surveillance. Prior to the vapor
release, outside contractors were allowed to install hardware without proper procedures and oversight.
Reference: Bowman
Once the SCAPE personnel were sure they had extinguished the flames on their suits, they evacuated
the area to the south. Both had received minor burns on their hands and were sent to a medical facility.
It was later noted that both SCAPE suits showed signs of burning when they were examined following the
incident. Shortly after the technicians entered the showers, two fire and rescue personnel who were
waiting on standby, entered the area and aided the water deluge in extinguishing the flames with their
charged hoses. It was reported that the fire was completely extinguished in about one minute.
Following the incident, there was an extensive inspection of the tanker and surrounding areas. It was
found that the fire was mainly concentrated around the shroud, which covered the sump flange, and the
stainless steel bucket. An exact cause of the fire was unable to be determined; however, several
possibilities exist. The following is a tabulated list of possible causes of the fire:
There was an extensive amount of corrosion (iron oxide) around the sump drain flange
The bucket was not properly inspected and cleaned of any possible iron oxides or other
contamination prior to running the procedure
The bucket could have heated up from solar radiation (the weather for May 16, 1984 was a high
o o
of 80 F and a low of 69 F)
There was a potential for a static charge buildup from the falling column of liquid
o It was reported that the tanker was grounded properly
o The stainless steel bucket used to drain the residual amount of MMH was not grounded
There was non-compatible rust-proofing undercoating on the wheel fenders of the tanker
The method of the flange removal was suspect
o One bolt was left in the flange allowing it to rotate out of the flow path freely while
spraying MMH in a fan-like pattern increasing the surface area wetted by liquid MMH and
introducing large amounts of MMH vapor into the air
It is unknown whether the fire started on the tanker structure or in the bucket. The SCAPE technician did
state that he did not feel the heat on his hands until after the residual MMH had been emptied from the
tanker into the bucket. Draining of fuel without knowing that an ignition source was present was the
proximate cause of the incident. Improper configuration management (maintenance of the hardware to
prevent the buildup of rust) and an improper training of personnel (flange removal) are noted as the root
causes of this incident. Following the fire, improvements were made to the fuel storage area water
deluge system and the fuel tanker trailers. More safety showers were also added to the storage area.
Page 29
NASA/TP-2009-214769
18 Liquid Trap in Purge Adapter Flexhose MMH Spill (5/24/1985, KSC OPF1)
On May 24, 1985, while preparing for the removal of R4D (down-firing thruster on the Space Shuttle right
pod) from OV-099, there was an MMH spill in OPF1 on the 10-foot level west side. R4D was being
removed as a result of an in-flight anomaly related to a heater failure on the thruster. The manifold had
been drained of its propellant and it was thought that a purge through the MD348 line (which connects to
the manifold) and out the thruster nozzle purge adapter was sufficient to remove the liquid and most of
the fuel vapors. Figure 18-1 shows a purge adapter in a cutaway of a primary thruster nozzle. During the
part of the procedure where the purge adapter was being removed, approximately one cup of liquid MMH
spilled from the thruster and down onto the body flap (which is illustrated relative to thruster R4D in Figure
18-2). The body flap was partially covered with spill protection sheets. The OPF1 hypergol exhaust fans
were activated and the facility was immediately evacuated. It was recorded that some of the liquid MMH
had saturated the body flap tile filler bar. It is unknown if the tile had to be removed. Unlike N2O4, which
was known to breakdown Koropon primer (from the STS-2 N2O4 spill mentioned previously), the effect of
MMH on Koropon was not well known in 1985.
Figure 18-1: Space Shuttle Thruster Chamber Cutaway with Purge Adapter Installed
R4D
10’ Level
Body Flap
Page 30
NASA/TP-2009-214769
There are three possible proximate causes to this spill. A combination of two or more of the following
causes is also possible. The first involves an improper configuration in which the flexhose that was being
used for the procedure had previously been utilized for a similar operation on the left pod thruster L1U’s
manifold. This flexhose was several feet longer than the flexhose that was normally used for the right pod
manifold evacuations. It was routed from the R4D thruster purge adapter to the 19-foot level above, then
down to the eductor on the 10-foot level. This excessive length in hose may have allowed for a “u-
shaped” low point trap to be formed in which liquid MMH could have collected. The second possible
proximate cause involved a procedural error made by the engineers on console. The evacuation could
have created a pressure differential across the liquid MMH. When the eductor was deactivated and
removed, this pressure differential may have pushed the liquid into the thruster chamber. The third
possible proximate cause was the liquid had collected in the thruster chamber (see Figure 18-1) and the
purge was of insufficient duration to remove all the liquid MMH or the liquid trap may have prevented the
vacuum source from evaporating all the residual fuel that remained around the purge adapter. A
definitive proximate cause was never determined.
Eight personnel in the area of the spill received minor injuries from the inhalation of MMH vapors.
Following the spill and cleanup, the thruster was successfully removed at KSC, sent to and repaired by
the manufacturer (Marquardt Corp.), and then reinstalled in the same location on the right pod of OV-099.
It appears that this spill could have been prevented if the proper hardware had been chosen for the
procedure. Items of this nature should, ideally, be addressed in the written procedure. Part numbers,
purge specifics, and toxic vapor check (TVC) sampling techniques should have been specified for the
operation. Improper procedure control and authoring are the root causes of this incident.
19 STS-61C OV-102 SRB HPU Loading N2H4 Spill (12/8/1985, KSC Pad 39A MLP
Surface)
On December 8, 1985, approximately three gallons of N2H4 spilled from a Leer-Romec AHC on the SRB
HPU system. It was later found that the AHC nipple was unthreaded from the AHC body because the
nipple-to-cap threads had seized. Therefore, the technician in SCAPE was spinning off the GHC/AHC
assembly from the propellant line that was online to the HPU propellant tank and bulk propellant rather
that unthreading the GHC from the AHC (proximate cause). There was not a fire and the orbiter
(Columbia) was undamaged. The SRB was also undamaged. Most of the liquid fell into the flame trench
about 75 feet below. The N2H4 that collected on the MLP surface was wetted with a fine spray of water
o
shortly after the spill. The high temperature for December 8, 1985 was 72 F. If the temperature had
been much greater and/or any rust was present on the hot mobile launch platform (MLP) surface, ignition
of the hydrazine would have been more probable. At the time of the spill, Challenger was on Pad 39B
awaiting the launch for STS-51L. The first flight of the Space Shuttle with the AHC anti-rotation devices
installed on the SRBs was the STS-26R return to flight mission after the Challenger disaster. Improper
vehicle design is the root cause of this incident.
20 Inadvertent Dry Well Removal MMH Spill (1/21/1986, KSC Pad 39A Fuel Farm)
On January 21, 1986, a technician removed a temperature transducer dry well assembly from the three-
inch MMH return line at the Pad 39A fuel farm. This was followed by a 12-foot high geyser of MMH from
the open three-inch line. MMH contacted the left side of the technician’s face and his left forearm. He
went immediately to the safety shower then onto a local hospital once he was evacuated. He was
st nd
diagnosed with 1 degree chemical burns on his face and 2 degree chemical burns on his forearm. It
was estimated that approximately 100 gallons of MMH liquid and/or vapor was released. The seven
technicians located at the fuel farm (some completing other unrelated tasks) immediately evacuated the
Page 31
NASA/TP-2009-214769
area. Moments later, two technicians reentered the area from an upwind direction to activate the firex
deluge system. It is very likely that this prevented a large fire. The technicians did not know that the
deluge system could be remotely activated from the LCC by contacting them over the Operational
Intercommunication System (OIS) or by calling 911. A fire would have likely occurred immediately if the
temperature had been a normal Florida summer day; however, it was mid-winter and the outside
o
temperature was 63 F.
This incident took place during an annual calibration of temperature transducers and other components,
using an OMI. The technician was correctly informed by the engineer on console to remove a specific
temperature transducer from the MMH return line. The technician mistakenly removed the dry well and
temperature transducer assembly (shown in Figure 20-1) rather than the temperature transducer alone
(proximate cause). If the temperature transducer was removed properly, there would not have been a
spill. The following is a quote from the mishap report that explains the incident:
“The LSOC hypergolic mechanical technician actually performing the task of transducer removal
used incorrect shop practice and loosened the wrong nut. Instead of using a one-inch wrench to
retain the dry well in place while loosening the transducer lock nut that threads into the dry well,
he used the wrench to loosen the dry well. He then used his hand to unscrew the transducer
thinking he was extracting the transducer from the dry well while he was actually unscrewing the
transducer dry well assembly from the three-inch MMH line. A mixture of fluid and vapor MMH
immediately erupted from the line sending a column 10 to 12 feet in the air.”
Probe Sensor
Later the engineer on console told technicians to empty the drainage sump at the fuel farm. This
ultimately led to contamination of a large area of standing water around the fuel farm.
It can be stated that this incident should have never happened because the GSE should have been
vented and drained of all the residual MMH; however, the inherent design of the dry well assembly was to
Page 32
NASA/TP-2009-214769
prevent breaking into a hazardous system thus allowing for a reduced level of PPE and risk of an
incident. Granted, the design of the dry well could have been improved by adding a retainer to prevent
the entire transducer and dry well assembly from being removed together (a root cause). The primary
root cause of this incident was lack of training of technicians. Engineers have to depend upon
technicians to make the correct decisions at some point; otherwise the controls to prevent an incident (in
attempt to reduce the risk to nearly zero) become so cumbersome that they hinder the actual operation,
let alone the additional cost of such controls. This is a good example of a case in which engineers had to
make a judgment call as to the level of PPE and system configuration. With the dry well design in mind,
the judgment call appears to have been valid.
There were many lessons learned from this incident. These include:
Dry well retainers are needed and are now used on all hypergolic propellant dry wells
o This design flaw in the GSE is considered a root cause
Training of technicians for proper removal of components needs to be scrutinized
o This is considered the root cause
The system tubing was not properly vented to ambient prior to the spill resulting in a 6 psia driving
pressure behind the liquid MMH (a proximate cause)
o Hypergol systems are usually left with a blanket pressure to prevent air intrusion and
should not be required to be vented for this type of transducer replacement
The system tubing had not been completely drained prior to running the transducer replacement
operation (a proximate cause)
o Draining of residual propellant should not be required for a transducer replacement as a
result of the dry well
The procedure was considered non-hazardous; therefore, the PPE was not at a level in which
one would don for entry into a hazardous system
o It was believed that the hazardous system would not be compromised by removal of the
transducer
Proper testing of the spill area for contamination should be completed prior to emptying the sump
tanks to grade
21 Relief Valve R&R Oxidizer Farm N2O4 Vapor Release (7/29/1986, KSC Pad 39A)
On July 29, 1986, there was a large N2O4 (NO2) vapor release from the storage tank at the Pad 39A
oxidizer farm. The event was somewhat planned; however, it was not anticipated that the final quantity of
the release was going to be as much as what occurred. Prior to 1986, when the storage tank relief valve
was required to be removed and replaced annually, it was accepted that there was going to be a release
of approximately one gallon of N2O4 (NO2) vapor as a result of the removed relief valve tubing being
online to the storage tank ullage and open to atmosphere. The estimated vapor release following the
removal of the relief valve in this instance was approximately 28 gallons.
Prior to the relief valve removal, the oxidizer storage tank was vented down. Removal operations began
immediately following the venting; therefore, the propellant in the storage tank was not given the proper
time to cool to below vapor pressure. There was still approximately 5 psig of pressure in the storage tank
(which was visible from the remote console in the LCC) at the time of the valve removal (a proximate
cause).
Another proximate cause of the larger-than-expected release was the tearing of a SCAPE technician’s
suit. He was trying to install the new valve while the storage tank was violently venting oxidizer vapors
(as a result of the large backpressure that remained in the storage tank), when his suit brushed against a
nearby sharp object. He proceeded to leave the area when he noticed the tear in his suit. This delayed
the installation of the new relief valve until a new group of SCAPE technicians arrived at the oxidizer farm.
It was reported that the cloud “exceeded the pre-designated clear area of 700 feet downwind” exposing
Page 33
NASA/TP-2009-214769
several personnel to the toxic vapors. The venting oxidizer vapor eventually cooled the liquid in the
storage tank from the boil-off. This significantly decreased the violence of the heavy vapor venting.
Along with the first technician who tore his SCAPE suit, two other technicians were exposed to oxidizer
vapors. One noticed yellow stains on his undergarments following the removal of his SCAPE suit. The
other reported burns on his right hand later in the day. All the exposed technicians were sent to the KSC
Occupational Health Facility (OHF) and released back to work within a few hours.
An improper GSE design and operational human error were the root causes of this incident. About ten
months prior to this relief valve removal and replacement operation, a formal request had been submitted
by engineering to install a dual relief valve system onto the storage tank with a three-way isolation valve
to prevent a vapor release during component removal. This request was denied. Following the large
vapor release, the modification was implemented at both launch pads, greatly reducing the possibility of
any future hazard. This dual relief valve modification allowed for isolation during the relief valve annual
maintenance. The modification was also implemented on the fuel storage tanks at both launch pads.
Figure 21-1 is a photograph of the relief valve locations on the top of the N2O4 storage tank with a close-
up photograph of the relief valve three-way isolation valve modification.
Relief
Valves
Improper configuration management was the root cause of this incident. After the fire, it was found that
an o-ring was missing from the outlet adapter fitting at a check valve in the vent system allowing an
Page 34
NASA/TP-2009-214769
external leak. It is assumed that the vent system was free of residual propellants since it was purged at
35 psig; however, since the vent had been utilized during the drain, some residual fuel remained in the
lines. The following is a list of observations related to this incident:
Cleanliness levels in the areas surrounding hypergolic operations must not be ignored
o The OPF water deluge trench was not routinely cleaned or inspected
The toxic vent system was not leak checked routinely; however, vent systems are not usually leak
checked because they cannot be completely pressurized
Care needs to be taken when installing components in hypergolic ground support equipment
Reference: Johnson
23 N2O4 and Insulation Adhesive Small Fire (6/23/1988, KSC Pad 39B Oxidizer
Farm)
On May 25, 1988, at the Pad 39B oxidizer farm, technicians were completing an N2O4 sampling
operation. During the operation, the technicians detected a small leak at the interface between the
sampling valve and the tubing (see Figure 23-1). The sampling operation was delayed until the leak
could be fixed. The repair was finally scheduled to be worked about a month later during the afternoon of
June 23, 1988. The removal of some local insulation was required to fix the leak along with re-torquing
the fitting that connected the sample valve to the tubing. This was all completed that afternoon.
Sample
Port
Insulation-
Wrapped Tubing
At about 10:30 PM on June 23, 1988, the sampling operation was successfully repeated. This was
followed by the disconnection of the sampling flexhose from the sample port. When the flexhose was
removed, liquid N2O4 spilled (about two tablespoons) into the small well surrounding the sample port. It
was not uncommon to have a small amount of residual N2O4 remaining in the sample flexhose when it
was disconnected. The spilled N2O4 reacted with the adhesive used for the attachment of the insulation
to the tubing and ignited (proximate cause of the fire). The fire burned itself out in about 10 to 15 seconds
before the technicians in SCAPE could return with a fire hose. There were no injuries to the personnel
working the operation and the only hardware damage was to the insulation, which was replaced.
The insulation had been installed within hours of the sampling procedure; therefore, it was not given the
appropriate time to fully cure. The adhesive used to attach the insulation to the tubing was found to
Page 35
NASA/TP-2009-214769
contain volatile hydrocarbons. This allowed for a chemical reaction to take place between the adhesive
and N2O4, causing ignition. The root cause of this incident was operational human error as a result of not
allowing the insulation adhesive to fully cure.
24 STS-26R OV-103 N2O4 Tubing Leak on Vehicle (7/14/1988, KSC Pad 39B)
On July 14, 1988, about three days after the loading of N2O4 onto Discovery at Pad 39B for the STS-26R
Return to Flight mission, a small N2O4 (NO2) vapor leak was detected in the left OMS pod (LP04) of
Discovery. STS-26R was the first flight following the Challenger disaster on January 28, 1986. The leak
did not result in any injuries or hardware damage, but it did impact the launch date of Discovery. Along
with several other Return to Flight actions, the launch team at KSC was required to test fire the three
main engines on Discovery while it was at the launch pad. This was known as the Flight Readiness
Firing (FRF) and was scheduled to occur on July 26, 1988. The Space Shuttle Program decided to
complete the FRF regardless of the known oxidizer vapor leak; therefore, the engineering team assigned
to assess the leak had a couple weeks to study possible solutions, while other groups were preparing for
the FRF.
First, the exact location of the leak within the pod had to be found (a cutaway view of the Space Shuttle
OMS pod can be seen in Figure 24-1). This was difficult because the internals of the aft pods and
forward module are not meant to be accessed while the vehicle is at the launch pad. The location of the
leak had been isolated to the LRCS N2O4 system or the associated tubing through monitoring of the
instrumentation in the LCC. The leak was measured to be 0.26 psi per hour (about 800 standard cubic
centimeters per hour) and a TVC found the concentration of NO2 vapor to be 15 ppm at the vehicle vent
door. This later decreased to 3 ppm. An access door was removed on the pod’s outer skin and several
internal thermal blankets were removed. A long section of PVC pipe was then inserted into the pod to
function as a guide for a borescope. Using a borescope, the leak was finally isolated (after a six hour
search) to a ½-inch dynatube fitting on the MD224 leg (the N2O4 propellant tank ullage vent). A cross-
section view of a dynatube is shown in Figure 24-2 and a photograph of the leak location can be seen in
Figure 24-3.
Page 36
NASA/TP-2009-214769
Compressive
Forces Parallel
to Center Line
Load
Carried by
Outer Rim
Primary
Seal at
Inner Edge
Integral
Diaphragm
Top of Pod
Dynatube
Location
ARCS N2O4
Propellant Tank
The engineering team came up with the following solutions to resolve the leaking dynatube:
Fix the leak at the launch pad (this had never been attempted before due to access issues)
o Cut a hole in the aft bulkhead structure (1307 bulkhead) and top of OMS/RCS pod
Put a “clamshell” around the fitting and inject a sealing compound into it, or
Drain the N2O4 propellant tank, disconnect and polish the dynatube, or
Cut out and replace the dynatube fitting with a new one
o Send a “stopper” up the line from the QD to plug the line (35 to 40 feet of tubing)
Page 37
NASA/TP-2009-214769
Return the vehicle to the VAB, de-stack, and swap LP04 with LP01 from Atlantis in the OPF
o The orbiter aft pods are not able to be removed and replaced at the launch pad since the
removal and handling equipment does not exist for this type of operation and the pods
are obstructed by the MLP tail service mass (TSM), shown in Figure 24-4
Return the vehicle to the VAB, de-stack, remove LP04 from Discovery in the OPF, send the pod
to the HMF to be repaired, and re-install it onto Discovery once the repairs had been completed
o This would delay the launch about two months
Fly with the leak and fix it following landing
TSM TSM
Left Pod
LP04
After a week of deliberation, the “clamshell” was selected as it was the least invasive method to stop the
leak prior to launch and it could be completed at the launch pad. Furmanite FSC-6B was used as the
sealing compound as a result of it having the best compatibility characteristics with N2O4. The FRF was
finally completed on August 10, 1988 following a few hardware-related delays. The repair of the oxidizer
vapor leak began on August 17, 1988 with the cutting of two holes in the aft 1307 bulkhead (the bottom of
the payload bay with the orbiter in the vertical orientation) and the top of the pod beneath the bulkhead.
The dynatube’s safety wire was removed and a “clamshell” (shown in Figure 24-5) was placed over the
leak. The Furmanite sealing compound was then injected. The leak was eliminated; therefore, the Space
Shuttle Program cleared Discovery for launch, which successfully occurred on September 29, 1988, a few
weeks later than the original launch date of September 6, 1988. The delay was a result of multiple items
including the FRF test delays, return to flight SRB testing delays in Utah, and the N2O4 leak on LP04.
Page 38
NASA/TP-2009-214769
Clamshell
Halves
The proximate cause of the leak was found to be a scratched dynatube. This was discovered during
ground processing two missions later when the pod was removed in the OPF and sent to the HMF for
repair. It was determined that the scratch must have occurred either when the dynatube was
disconnected in October of 1987 to allow access to a nearby valve for a removal and replacement or
when it was reconnected in late December of 1987. Following the reconnection, the dynatube fitting
passed all leak checks. Transport vibrations may have “altered” the state of the connection between the
HMF, OPF, and launch pad; however, this is highly unlikely since dynatubes do not typically change
mechanical state, especially with safety wire installed. The Kersey report states that “The most probable
cause was that the mass spectrometer probe operator probed the wrong RCS propellant system (i.e.
MMH instead of N2O4).” Operational human error was the root cause of this incident along with improper
training. This categorization was given as a result of the incident being a consequence of either an
inadequate inspection of the dynatube sealing surface or an error made during the leak check of the
dynatube.
A question was raised concerning the accuracy of the mass specification leak check, which was the leak
detection method used on this dynatube in December of 1987. As a result, a new policy was
implemented to independently verify zero leakage using two separate methods. Currently, the Space
Shuttle hypergol systems utilize a hazardous vapor (“sniff”) leak check, in addition to the mass
specification leak check of individual fittings when they are disconnected for any reason.
The Kersey report also lists the following as findings and recommendations to reduce the risk of an
undetected dynatube leaks in the future (below some items are resolutions to the findings or
recommendations):
Page 39
NASA/TP-2009-214769
o The existing mass spectrometer units are complex to operate, outdated, and vulnerable
to transportation damage.
Six new Leybold mass spectrometer units were procured.
o Perform mass spectrometer leak tests of all joints in lieu of rough pressure decay checks,
when systems have been decontaminated/cleared of hypergolic propellants.
o Revise procedures to include data sheets, requirements, calculations, and equipment
verifications to assure accurate and consistent data is recorded for mass spectrometer
leak checks.
Procedures were revised.
o Procure new tools, develop repair standard and inspection procedures to the latest
criteria, and develop a standard training certification program for all personnel working
with dynatube fittings.
Two new repair kits were procured.
Standard OMI’s were developed for dynatube inspection and repair.
The inspection specification was updated with specific criteria.
A formal certification was established for working with dynatube fittings.
o Form a well-trained and dedicated mass spectrometer maintenance team authorized to
do repair work.
An engineering “focal point” was established for controlling all efforts with
dynatube fittings.
One possible cause of the fire is the NaOH(aq) + HCl → NaCl + H2O reaction, which is highly exothermic.
It is possible that residual MMH or N2H4 dissociated from solution while flowing into the containment tank
and collected in the ullage to later ignite from the heat generated by the addition of the HTH into solution.
This, however, has not been proven.
Reference: Henderson
Page 40
NASA/TP-2009-214769
When the service valve SV12 (incorrectly labeled as SV11) was opened, hypergolic reactions began to
take place in the fuel aspirator flexhose and downstream vent lines. The oxidizer manifold pressures
dropped from 80 psia to 22 psia and the technicians noted that the flexhose was warm to the touch. The
service valve SV12 (really SV11) was immediately closed. Later, the flexhose was removed and flushed
with water along with the fuel aspirator. It was approximated that a maximum of 2 tbsp of N2O4 may have
been aspirated. The technicians noted some discoloration in the flow direction identification tape on the
fuel aspiration lines. It is thought that most of the reaction took place in the flexhose and first 25 feet of
the TS 401 facility line.
Improper configuration management (improper labeling of components) was the root cause of this
incident. Some possible improvements to the test setup include keying in the N2O4 and MMH fittings to
preclude mismatching and improved labeling and redundant labeling of components.
Reference: Meyer
27 HMF Screens Test Drum MMH Spill (12/7/1990, KSC HMF M7-961 East Test
Cell)
During the first aft reaction control system (ARCS) propellant tank screens test at KSC’s HMF M7-961
East Test Cell on December 7, 1990, roughly one to two gallons of liquid MMH spilled from the relief
valve of a 55 gallon MMH drum and onto the floor of the test cell. The amount of liquid MMH, sufficient to
cover the lid of the 55-gallon drum, represented a significant surface area once it spilled to the HMF cell
floor. Experience and research have shown that the likelihood of an MMH flash fire is directly
proportional to its surface area. It was fortunate that this incident did not result in a fire, which likely did
o
not occur as a result of the conditioned air in the test cell being about 70 F and the floor area being clear
of any iron oxide. Once the source of the leak was determined, the manual fill valve was closed, the
drum was pressurized, and the liquid was drained back into a nearby tanker trailer.
It was determined that a hard-line hose from a technician’s SCAPE suit caught on a manual fill valve
handle and opened it, filling the drum from the tanker (proximate cause). The handle was sticking out
beyond the edge of the panel. Following the spill, the handle was removed and rotated 180 degrees so
that it no longer protruded beyond the edge of the panel. Entry into the cell was not allowed for several
days since some of the liquid MMH ran into areas that were not easily accessible. The root causes of this
incident were improper GSE design (protruding handle) and human error (lack of technician situational
awareness). One must consider, however, that a technician’s visibility in a SCAPE suit is poor, especially
peripheral vision.
28 STS-42 OV-103 Ferry Plug Removal MMH Spill (2/12/1992, KSC OPF3)
Following the STS-42 mission and ferry flight of Discovery from EAFB on February 12, 1992,
approximately ¼ to ¾ cup of MMH spilled from a RCS thruster nozzle. This occurred when the ferry plug
was removed in OPF3 (proximate cause). The liquid contacted the gloves and right shoe of the
technician who was removing the plug. There was no reported hardware damage. The collection of
liquid in the thruster was believed to be a result of cold temperatures causing the thruster valve to leak
(see section 14). The vehicle was not powered during at least one stopover for refueling of the SCA;
therefore, the thruster heaters remained off. At this point in the Space Shuttle Program, the limitations
and sensitivities of the thruster valves were well understood. The root cause of this incident was
improper design as a result of not implementing provisions to provide ground power during the SCA
stopover.
Reference: RCS Thruster Heater Power during Ferry Flight Stopovers; Heinrich
Page 41
NASA/TP-2009-214769
The appearance of a ¼-inch AHC cap is such that it could be mistaken for a ½-inch AHC cap to
an untrained technician
o The technician was undergoing on the job training under the direct supervision of an
experienced technician, but the distinction between a ¼-inch cap and a ½-inch cap was
not explained to him
Age and wear made part number identification on the AHC cap difficult
Work instructions should be written to include positive identification of interface equipment
AHC cap mating should not be considered a benign activity; therefore, PPE should be used to
protect the employees from credible releases of propellants
Future designs should consider keying of AHC caps to preclude incorrect size/commodity
installation
0.235 inches
Figure 29-1: ¼-inch and ½-inch Hypergolic Flight Cap Cutaway View
Reference: Reynaud
30 Thermochemical Test Area N2O4 Vapor Release (4/21/1994, JSC Building 353)
On April 21, 1994 at the JSC Thermochemical Test Area (TTA) Building 353, there was a large N2O4
vapor release at approximately midday. Engineers and technicians were preparing for a simulated
altitude test of a small bi-propellant rocket engine. Oxidizer was first flowed in the system on February
22, 1994; however, a few leaks were noted and the system was drained and refilled twice following
troubleshooting of the noted leaks. During the thruster firing procedures on April 21, 1994, a three-way
solenoid valve failed allowing liquid N2O4 to leak into the vent line and oxidizer burner stack (proximate
cause). It was later determined that the valve had an extra one-half of a Teflon seal, extensive corrosion,
and some small scratches on the armature. It is unclear which particular item caused the valve to leak,
but the most likely candidate was the extra one-half of Teflon seal.
The test team was unaware of the valve failure and continued on with their procedures. They first
brought up pressure in the oxidizer tank to approximately 850 psig, which was greater than the desired
Page 42
NASA/TP-2009-214769
pressure of 800 psig. The solenoid valve leak rate then increased filling the burner stack with N2O4 liquid
and vapors. The over-pressurization required the test team to vent approximately 50 psig off of the
system. When they proceeded with the venting process, a small amount of oxidizer vapor exited the
burner stack. This was an indication that there was a problem with the system; however, the test team
proceeded on while rationalizing the small oxidizer cloud as an incorrect methane burner supply valve
setting.
The test team decided to continue on with their procedures even though there was an inadvertent oxidizer
vapor release and some data in the control room was indicating possible issues with the system. It is
unknown if the test team simply did not notice the data or they misinterpreted it. A short amount of time
passed while the test team assessed the situation allowing the oxidizer tank pressure to drop (as a result
of the leaking three-way solenoid valve) to 600 psig. The test team failed to notice the significance of the
200 psi pressure drop and proceeded to increase the oxidizer tank pressure again to approximately 800
psig. When they did this, there was a large N2O4 (NO2) vapor release from the burner stack
(approximately 16 to 20 gallons of liquid N2O4 and NO2 vapors). The oxidizer cloud traveled downwind in
a westerly direction resulting in the displacement of dozens of people. About 81 people (9 NASA
personnel, 70 contractors, and 2 visitors) received minor injuries from inhalation of the oxidizer vapors.
The following are contributing factors and/or lessons learned from this incident:
A formal emergency response did not exist for this type of event
o The emergency response personnel at JSC lacked the proper training and practice for a
hypergol spill, vapor release, or fire at the time
The process lacked proper identification of safety control hazards and emergency procedures
There were insufficient controls in the system as it contained single point failures (improper
design; a root cause)
The system should have contained a bypass relief valve that could withstand liquid lock (improper
design; a root cause)
Several attempts were made to tighten leaky fittings and clean up liquid oxidizer spills without the
use of any PPE
The test team failed to notice the severity of the events as they were unfolding by either the data
that was being obtained in the control room or the initial small cloud of NO2 vapor that was
released; lack of situational awareness or human error was one root cause of this incident
The test team lacked the proper training for dealing with hypergols (another root cause)
JSC management was unaware of the risks in preparing for and conducting the test of this bi-
propellant rocket engine
There was a lack of voice and video recording of this operation
Facility changes were not processed correctly and were not distributed to the appropriate
disciplines; improper configuration management was another root cause of this incident
KSC management also felt the need to assess their own safety practices following this incident
o GSE and facility designs, training requirements, emergency response procedures, and
procedure review practices were reviewed and updated accordingly for all hypergolic
systems at KSC
Page 43
NASA/TP-2009-214769
when the solar heat caused the pressure to increase in the N2O4 tubing while the system was in a static
configuration.
TVC N2O4
Propellant Tank
The pressure increase went unnoticed by the engineers and technicians. Remote data monitoring was
available on the launch vehicle, but not on the propellant transfer systems (GSE). Local dial gauges likely
showed the increase in pressure; however, there were no technicians monitoring these particular gauges.
The pressure rise eventually caused a weld seam to fail at the base of the transfer system (proximate
cause). This portion of the N2O4 servicing GSE lacked a relief valve, thus allowing the propellant in the
tubing to become “liquid locked” since a drain-back and purge had not been completed (improper design;
a root cause). There was no damage to the GSE (excluding the ruptured tubing) or vehicle hardware and
there were no reported injuries.
Usually this tubing would not contain any propellant unless a loading operation was underway; although,
schedule pressure led to the decision to halt the propellant drain-back and purge following the loading.
The processing schedule had been delayed by a couple hours as a result of addressing a leak on a fill
and drain valve on the SRM TVC system. Once the fill and drain valve leak was repaired, other non-
related, time-critical work required attention on the upper stage of the Titan IV rocket. As a result, the
Launch Conductor delayed the purge of the oxidizer loading system by several hours since these two
operations could not be conducted in parallel. Once the upper stage task was completed, the Launch
Conductor delayed the purge again to allow the engineers and technicians to prepare for the fuel loading
operation. Fifteen minutes after the vehicle fuel tanks were loaded for flight, the N2O4 GSE tubing
ruptured. The release amount was approximately 350 to 400 gallons into the launch pad transfer slip.
Another root cause of this incident was determined to be frivolous procedure deviations. The loading
team and Launch Conductor had experience with successfully delaying the oxidizer drain-back and
purge; however, this experience was based on insulated oxidizer tubing. The tubing at SLC-41 had
recently been removed and replaced with non-insulated tubing as a result of the moisture trapped in the
insulation causing accelerated corrosion. The loading team and the Launch Conductor either did not
recall this or they were never told.
There are requirements for launch Pads 39A and 39B to have seamless pipe throughout and to have
relief valves in tubing that could become “liquid locked.” Following the incident at SLC-41, the procedures
were changed to preclude this sort of incident from happening in the future. The procedure deviation
approval rules were also reaffirmed to the loading team and Launch Conductor.
Page 44
NASA/TP-2009-214769
Figure 31-2: Titan IV A TVC System Schematic (see Table 31-1 for references)
Page 45
NASA/TP-2009-214769
Right
Stinger
Left
Stinger
Prior to the reported fire, the technician removed the Aclar (plastic film) bag from the open end of the
thruster feedline dynatube fitting. This fitting connects the manifold to the thruster. When he did this, he
noted that there was an exceptionally large amount of fuel vapor present and some liquid MMH was
dripping from the open line. The technician proceeded to place the aspirator directly on the feedline
dynatube (the procedure and known practice was to place the aspirator perpendicular to the dynatube
exit). This was likely the proximate cause of the fire. The engineer on console then noted that the
manifold pressure dropped from 16 psia indicated to 6 psia indicated in about five seconds (the
transducer had a 2 psi offset). The manifold pressure stabilized to 16 psia indicated after a couple
minutes. At this point, the technician was instructed by the engineer to remove the aspirator from the
dynatube. He did this and then left the area to find a mirror to aid in the dynatube inspection. When he
approached the dynatube with the mirror he noted a “cream/yellowish color” flame and “heat waves.” As
previously mentioned, he proceeded to put out the flame with his glove after the other technician doused
the area with alcohol.
The engineer on console was not convinced that the technician had seen a flame because it was his
belief that MMH fires are not visible or are clear (as noted in section 1.3 Properties of
Monomethylhydrazine (MMH), MMH flames are usually light yellow in color). He thought that the
technician had seen some off-gassing of the MMH that was enhanced by a nearby light source; however,
this cannot explain the flash fire from the alcohol spray. This flash required a flame to already be present.
Page 46
NASA/TP-2009-214769
The engineer voiced his thoughts to management and they trusted his opinions as the system expert.
The technician who reportedly put out the fire with his glove, later made a statement that he was
instructed to “not talk about flames or fires over the net” by engineering and technician operations. If this
did actually occur, it is very unfortunate that the technician was specifically told to not follow the
appropriate safety protocol in the event of an incident.
It was later determined through analysis that an MMH decomposition (exothermic) reaction had occurred
in the thruster feedline and there was likely a flame present. When the aspirator was placed onto the
dynatube, it removed the inert GN2 from the line and replaced it with an inrush of air. The residual MMH
that remained as a thin film around the inner diameter of the tube began to react with the oxygen in the
o
air. When the decomposition reaction reached a temperature of 382 F within the interior of the feedline,
ignition occurred. A flame will persist until the fuel is exhausted or the oxidizer is removed or exhausted.
In this case, the glove of the technician’s SCAPE suit removed the oxidizer.
No damage to the tubing or to other hardware was found during the inspections immediately following the
fire. The PR closure stated the following as a possible reason for ignition:
“…the ignition was the result of a vacuum being inadvertently placed on the line by a technician
using a spill aspirator. This vacuum removed the inert GN2 which was present inside the line.
When the aspirator was pulled away, it allowed air to enter the line. The air, which contains
oxygen and carbon dioxide, reacted with the thin layer of fuel [MMH] residue inside the tube. This
o
reaction continued until the fuel [MMH] has reached the flammability limit of 382 F.”
Possible annealing of the feedline became a concern. This was addressed in the original PR in which the
closure stated the following:
“…assuming the worst case where the tubing was annealed during the fire, the maximum
pressure (approximately 1200 psia) the line could have possibly seen was well below (less than
50%) the yielded strength of 5/8-inch diameter, 0.028-inch thick 21-6-9 cres [corrosion resistant
steel] tubing in an annealed condition.”
Page 47
NASA/TP-2009-214769
AP59-02
Door
OME Nozzle
Exit
RP01 Fire
AP59-02
Door
OME Nozzle
Exit Cover
Right
Stinger
Page 48
NASA/TP-2009-214769
A technician in SCAPE was disconnecting the R1A thruster MMH feedline dynatube during a thruster
removal and replacement procedure. The procedure was initiated following an in-flight anomaly that
occurred on the thruster during on-orbit operations (R1A primary thruster failed off due to a low chamber
pressure). The N2O4 dynatube had already been disconnected and double-bagged in attempt to contain
the residual vapors that were being released from the fitting. The technicians were told to place an
aspirator near the N2O4 dynatube and proceed with the fuel feedline disconnect operation. When the seal
of the fuel dynatube was broken by the technician using a wrench, he noted that there were “large drops
of fuel being siphoned away by the aspirator.” Immediately following this, a fire erupted. There were six
SCAPE personnel in the area when the fire was first reported. The proximate cause of this incident was
the disconnection of the dynatube without knowing that there was liquid fuel within the tubing.
The technicians managed to put the fire out with a parts rinse bucket nearby, which they had to refill
several times with water in the safety shower. Apparently, there was not a charged water line in the
vicinity. Once the fire had been reduced in size from a large flame to a small flame at the open end of the
dynatube, a technician cupped the end of the dynatube with his SCAPE suit glove and placed the
aspirator in the flame, which immediately put out the fire.
The engineers on console in the LCC believed that they had purged the manifolds enough to reduce the
residual MMH to manageable levels. Immediately preceding the feedline dynatube disconnect, the
engineer on console incorrectly read the fuel feedline pressure transducer on the vehicle. The subject
transducer indicated 16 psia (approximately atmospheric pressure). In actuality, the transducer was
biased low (as determined from a previous test completed during the OMDP of RP01 about 18 months
prior); therefore, the actual pressure in the manifold had increased to about eight psi above atmospheric
pressure. There had also been a shift change between the venting and disconnection operations which
may have led to the transducer bias going unnoticed. The increase in pressure may have been a result
of a leaking manifold one isolation valve (LV317, the valve holding back the bulk propellant from the
feedline).
Another possible scenario (not included in the formal mishap report) for the pressure increase was put
forth stating the pulse purges were unable to remove all the liquid MMH as a result of a low point trap
present in the manifold near the MD317 (manifold 1 drain/purge QD) AHC interface. Figure 33-3 shows
the basic layout and low point in fuel manifold 1 in the right pod stinger area. It is believed that during the
purge, gas was unable to enter the line through the MD317 QD to displace the liquid MMH out of the
manifold. This was a result of a small bubble that had formed in the low point as a result of helium
coming out of the liquid MMH/helium solution. The helium slowly coming out of solution may have been
the source of the pressure rise in the manifold rather than a leaking manifold isolation valve. Once the
fire started, it heated the area thus increasing the pressure and pushing additional fuel out of the fuel
feedline that was trapped. In both scenarios (leaking manifold isolation valve or helium coming out of
solution), liquid MMH was present in the manifold line just prior to and during the fire and the manifold
pressure increase went unnoticed.
Page 49
NASA/TP-2009-214769
The “official” cause of the ignition from the mishap report was stated to be electrostatic discharge passed
from the materials (Orcofilm being one of them) used for spill protection to the SCAPE suits, gloves, and
tools. The spill protection materials and SCAPE suit failed electrostatic testing performed following the
mishap. Another possible cause for ignition (not put forth by the mishap report) was a decomposition
combustion reaction of the MMH and a material that it contacted. A hypergolic reaction with the N2O4
(NO2) vapors that were possibly leaking from a nearby double-bagged oxidizer feedline dynatube was not
a likely source of ignition since the aspirator was positioned directly adjacent to the open end of the
oxidizer feedline. An atmosphere of approximately 10% N2O4 (NO2) vapors is needed for a hypergolic
reaction with MMH vapors in air. With the amount of atmosphere that the aspirator was drawing in and
the fact that the feedline was not receiving a constant pressurized source of N2O4, obtaining a 10%
atmosphere of N2O4 (NO2) vapors was not possible.
About six months to a year prior to the fires on left and right pods of OV-105, a Tiger Team (an
independent investigation team) was investigating on-orbit thruster failures. One finding from the study
was that evacuation of the thruster feedlines to remove residual propellant prior to a thruster removal and
replacement was detrimental to the health of the thruster valves. Therefore, the thruster removal and
replacement procedures were changed to disconnect the thruster feedlines utilizing a pulse purge rather
than an evacuation, also known as a “hot” removal and replacement. Dozens of thrusters were
successfully removed and replaced without incident following this change. The inadequacy of the pulse
purges in removing all of the residual fuel in the feedline and the unnoticed pressure increase in the
manifold from either a leaking manifold isolation valve or helium coming out of solution (operational
human errors) were the root causes of this incident, but other unknown factors likely played a role.
Following the fire, the oxidizer feedline was plugged and the fuel feedline fitting was tightened to safe the
system. There were no reported injuries. There was an appreciable amount of damage to flight
hardware from the fire and the water that was used to extinguish it, including several RCS thrusters, three
purge ducts, some electrical harness ties, and eight thermal control system (TCS) blankets. This damage
required removal of the OMS pod for refurbishment at the HMF. A total of 33 PRACA items were taken
for damage as a result of the fire. The three purge ducts, eight thermal blankets, and four propellant line
clamps were removed and replaced with like items along with thrusters R1R, R1U, R3A, R3R, R3D, and
R1A (which was removed in the OPF immediately following the fire). The only thrusters that had fire
damage were R1R, R1U, and R3A, the rest were required to be removed since they were located on the
same manifold as another thruster that was being removed. The thrusters were refurbished and returned
to logistics as usable spares.
The delay as a result of the repair from the fire was an impact to the processing flow of the pod; therefore,
two pods were swapped between orbiters. OV-102’s RP05 was re-assigned to OV-105 and RP01 went
to OV-102, after repairs. RP05 had just come out of its OMDP.
There was not one simple reason for this incident. As with most accidents, many different, seemingly
unrelated items created the exact environment needed to support a fuel fire. The following items were
noted as lessons learned from this incident:
Operational procedures did not adequately safe the system for hazardous operations (a root
cause)
System hardware design forced complicated thruster removal and replacement operations
o Tiger Team results led to an increase in hazard during the thruster removal and
replacement operation
Designs should be completed with operability in mind
o The thruster feedline dynatubes in the FRCS allow for independent disconnection of the
MMH and N2O4 lines, where the ARCS feedline dynatube bellows force the disconnection
of the MMH and N2O4 lines simultaneously (the orbiter FRCS and aft pods were designed
and built by different companies)
Understand and design to mitigate external/environmental impacts
Understand failure modes during all phases of operations
Page 50
NASA/TP-2009-214769
Two valve isolation is the desired configuration (if possible) during an operation involving
hypergols being opened to the atmosphere
Training of the technicians that performed the operation was found to be inadequate (another root
cause)
Fire fighting equipment should be nearby when an operation involving the opening of a hypergolic
propellant line to atmosphere is being performed
o Walkdowns prior to the operation failed to point out the lack of fire fighting equipment
o It should be noted that SCAPE personnel are not required to stay and fight a fire;
however, historically most stay in attempt to contain the situation
There was a crew located at the ORSU to collect a sample of the N2O4 in the tank since it was being
circulated on the same day to filter out particulate. Prior to this sample being taken, the primary oxidizer
burner was set to 100 percent by the blockhouse monitor to support venting of N2O4 (NO2) vapors from
the ullage of the storage tank to ambient pressure. The TS 405 propellant tank venting occurred shortly
thereafter. The personnel in the vicinity were not exposed to the liquid N2O4 or NO2 vapors and were
immediately evacuated to a safe area. It was later determined that a contributing factor was that the 400-
Area vent system was not an active system; therefore, it did not receive the full engineering design review
as did the pressurized (active) system (a root cause). It was also found that the 400-Area configuration
was not properly managed (a root cause).
The technicians were performing a modification to the MMH eductor system on the 10-foot left platform of
OPF2. The modification included the installation of permanent hardware by removing flexhose interfaces
and installing hard-line tubing, adding a suction port, and raising the eductor panel from the floor and
attaching it to a railing. The last time the eductor system was used for an operation on the orbiter was
about two years prior, in which MMH was drained from the LRCS manifold two of Discovery for the later
removal of thruster L2D. It was later determined that the vent line had only been “blown down” rather
than educted with a vacuum; therefore, liquid was able to collect in the low points, which should have
been designed out of the system (improper GSE design; a root cause).
With the technicians in supplied air respirators and hypergol compatible aprons and gloves, the first part
of the modification, which consisted of installing a new valve and tee assembly for the additional suction
port, was completed without incident. No liquid MMH was noted as spilling even though the same 1-¼-
inch vent line from which the spill occurs a couple weeks later, was opened. A reading of 0.8 ppm was
Page 51
NASA/TP-2009-214769
noted at one of the capped fittings on the tubing, but this was not out of the ordinary. The 1-¼-inch vent
line was then opened a second time to install the modified 1-¼-inch line that connected the new valve to
the vent line. Minor vapors were detected when the 1-¼-inch vent line cap was removed. The
technicians found that the modified line was too short and some other lines were installed incorrectly, so
they re-installed the 1-¼-inch vent line cap and reported the problems.
When they returned to fix the installation a few days later, a few droplets of MMH dripped onto one
technician’s glove and arm when he initially loosened the cap. He quickly rinsed off his arm in the nearby
safety shower. The safety technician then proceeded to take interscan readings of the fitting, which he
reported were 0.0 ppm. Since the readings were found to be 0.0 ppm, the technicians determined that
the liquid was probably alcohol (used for cleaning the fittings), water, or condensate. They all then
decided that it was safe to continue. Following this, the technician began to remove the overhead 1-¼-
inch vent line cap when liquid MMH spilled out onto his and the other technician’s skin and the
surrounding GSE. Another safety technician nearby notified the pad leader at the OPF operations desk
to evacuate the OPF and activate the hypergol fans to remove the residual fuel vapors. The area warning
“warbler” was activated and proved to be ineffective at getting all of the personnel out of OPF1, OPF2,
and the OPF annex; therefore, the fire alarm was activated. This deactivated the hypergol exhaust fans.
Why the area warning “warbler” was ineffective is unknown.
The exposed technicians proceeded to take turns rinsing off in the safety shower while fully clothed and in
their supplied air respirators. They then began their evacuation of the facility; however, their supplied air
hoses became entangled forcing them to remove their masks while still in the MMH vapor cloud. Once
they had exited the facility, paramedics on site began to examine them inside an ambulance. One
paramedic indicated that the technicians had a fuel odor; therefore, they were instructed to remove their
clothes so they could be rinsed off by fire personnel. They were then given blankets and transported to
the OHF in two separate ambulances. One technician was told by paramedics to report to the OHF using
his own personal vehicle. When he arrived at the OHF his clothes were found to be contaminated with
fuel. He was told to shower and then put his contaminated pants back on once they were found to be 0.0
ppm. About 30 minutes later, he removed the contaminated pants again as a result of a fuel odor, and he
along with the other technicians were given coveralls. When fuel exposed clothing is rinsed with water,
the water can conceal the residual fuel in the clothing until the water evaporates. This is likely why the
residual fuel went unnoticed in the technician’s pants resulting in him being asked to put his clothes back
on after showering at the OHF. Per safety guidelines, the clothing should have been treated as
hazardous waste.
The reason for the spill was an unknown low point in the 1-¼-inch vent line which trapped liquid MMH.
The reconfiguring of the lines allowed the low point to move around within the line and go unnoticed. It
appears that the technicians reacted as best they could with the tools they had with the exception of not
removing their clothing in the safety shower. The interscan fuel detector used during the operation was
found to have a faulty pump. The pump was contaminated with liquid MMH and only worked
intermittently; although, the interscan did read full scale high during the spill along with its alarm sounding.
This could be the reason for the 0.0 ppm reading when the technician initially opened the cap; however,
the pump could have been contaminated during the spill. There were several lessons learned following
this incident including:
The fire alarms deactivated the hypergol fans and should not be used during a hypergol related
evacuation
o Following the incident a by-pass switch was added to the heating, ventilation, and air
conditioning system outside each OPF so the hypergol fans can be turned on when a fire
alarm has been activated
o A fire alarm should not be used to evacuate personnel following a propellant spill
o All processing facility personnel should be familiar with evacuation alarms
The recommended decontamination procedure was not followed in which the person’s clothing is
to be removed prior to entering the safety shower and the clothes are to be treated as hazardous
waste
Page 52
NASA/TP-2009-214769
o Following this incident “modesty garments” were placed in boxes near the safety showers
throughout KSC
o Curtains were also added to the safety showers
After being involved in a serious incident, personnel should immediately seek medical attention
and should not be asked to drive themselves to the medical facility
The medical personnel did not seem to be knowledgeable as to the proper treatment of a person
who has been exposed to hypergols
An improper GSE design can destine a system for failure
o Low points in GSE should be designed out (root cause)
Reference: Stefanovic
36 HMF Sample Valve MMH Spill (3/26/1997, KSC HMF M7-1212 West Test Cell)
On March 26, 1997, in the HMF M7-1212 West Test Cell, an MMH spill occurred from a GSE sample
valve. FRC4 had been removed from OV-104 as a result of a suspect thruster dynatube seal saver
installation. The dynatube disconnection operation required the thruster manifold to be drained and
educted for no less than eight hours. Following this, preparations were made to perform a sampling
operation. Three technicians set up the 20-foot safety clear area. One technician remained outside the
clear and was not wearing any PPE. The other two donned hypergol compatible aprons and gloves and
wore supplied air respirators.
One technician powered up the aspirator and then began to remove the sampling valve outlet cap from
the MD137 flexhose (see Figure 36-1 for a close-up view of the sampling valve). A TVC of the area was
then performed with a resulting reading of 0.0 ppm. The technician laid the aspirator hose on the side of
the rack, which was not a suitable position to capture any liquid or vapor that may be released from the
valve outlet as the procedure stated.
AP28-00 Door
FRCS
Sample Valve
Page 53
NASA/TP-2009-214769
The technicians were then at the point in the procedure where the valve was to be opened. The
procedure was written to imply that there may be some vapor released from the valve when it was
opened. Liquid MMH was not expected, but, as it turns out, there was a liquid release once the valve was
opened. MMH splashed out of the sample valve and onto one technician’s left arm, the glove of the other
technician, and the surrounding area. It is estimated that approximately ¾-cup of liquid MMH spilled from
the valve. The technician immediately closed the valve and began to evacuate the test cell along with the
other technicians. He exited the test cell and located a safety shower, which he began to use. A medical
and fire team were immediately dispatched to the scene and the engineers on console notified the proper
authorities. The technician did receive chemical burns on his arm for which he was treated at the OHF.
He was released within a few hours.
The source of the liquid MMH was a low point in the flexhose, which retained some of the liquid MMH
following the eductions (proximate cause). How the liquid MMH was able to enter and collect in the
flexhose low point is more complicated. The post-incident data analysis showed that the manifold drain
and eduction were quite typical with no out-of-the-ordinary anomalies. However, this was the first time (at
least in the memories of the engineers working the procedure) that a manifold had been drained in which
the MMH was not saturated with helium. Previously, during a separate operation, the manifold had been
drained and later backfilled with liquid propellant that was not saturated with helium. Helium had not been
introduced into the system since the module had not flown following that operation.
When helium saturated MMH is vented to ambient and drained from a manifold, the helium that was
absorbed into the fuel comes out of solution and forms bubbles which rise to the high point of the system.
The bubbles expand and act as an expulsive force while the system pressure is decreasing. This helps
to “push” the liquid MMH out of the manifold. When unsaturated MMH is drained, a capillary action takes
place (this is similar to holding a finger on the end of a liquid-filled straw). The MMH does not drain from
the manifold as easily and, over time, it collects at the low points as a result of in-specification leakage of
the upstream manifold isolation valves. With the system pressure at approximately 23 psia, the trapped
liquid fuel in the low point of the flexhose was forced out to atmosphere when the sample valve was
opened. Operational human error was one root cause since the engineers were unaware of the effects
that the lack of helium saturation in the manifold would cause during the drain. The following are findings
and observations from the incident report:
The flexhose had a low point in it; this should be eliminated though proper flexhose routing
o The manifold had an inadequate drain and eduction to remove all of the liquid MMH
o Operational human error (a root cause)
The test team did not completely understand the ramifications of draining a system which
contained unsaturated MMH
The aspirator was not positioned properly at the valve when it was opened, as the procedure
stated
The procedure did not identify the potential for liquid MMH being released from the sample valve
o There was no method to verify that the manifold did not contain any liquid
Reference: Sullivan
Page 54
NASA/TP-2009-214769
TVC N2O4
Propellant Tank
About 20 minutes after initiating the propellant tank pressurization, liquid N2O4 was observed to be
streaming out of the aft skirt heat shield region on both SRMs. There were no reported injuries and the
flight hardware had minimal collateral damage (mainly on the SRM nozzles). At the time of the observed
N2O4 release (likely initially noticed by the signature brownish-orange NO2 vapors characteristic of an
N2O4 spill) the propellant tanks were at a pressure of about 830 psi. The target pressure was about 1,040
psi. The engineers quickly began to depressurize the propellant tanks, which had the immediate affect of
decreasing the rate of propellant release. The tanks were vented from their initial pressure of about 830
psi to a final pressure of 38 psi, at which time the N2O4 release ceased. At that time, all systems were
secured and the operation terminated. It was decided to offload all of the propellant from the subject
tanks, so about eight hours later, the engineers and technicians began the offloading procedure. This
operation was completed without incident. Following the offload and evacuation of the TVC N2O4 system
and tubing, the total amount of propellant transferred was estimated. About 1,160 gallons of N2O4 was
re-captured as a result of the drain-back. This logically implies that about 244 gallons of N2O4 was
released to the atmosphere.
The source of the leak was found to be the electronic compartment purge ports on two flow control valves
(FCVs) on SRM #1 and one FCV on SRM #2. A cutaway view of the FCV is shown in Figure 37-2.
These valves (along with the one remaining valve that did not leak) were removed and sent to the
manufacturer for failure analysis. Upon disassembly of the three failed valves, evidence of combustion
was noted at the electrical compartment’s omniseal. The omniseal had been extruded through the
sealing surface and the retainer ring had been ejected. The degradation of the omniseal resulted in liquid
N2O4 flowing into the FCV’s electronics compartment and out the purge port. There were no leaks found
within the TVC propellant tank or the associated system plumbing.
Page 55
NASA/TP-2009-214769
Failure analysis determined that the proximate cause of the seal degradation was traced to a chemically
reactive incompatibility of N2O4 with the cleaning agent, Bio T-200A, used on the FCVs during
manufacturing at that time. Freon had long been the industry standard for oxidizer component cleaning;
however, in 1987 the international scientific community endorsed the Montreal Protocol Treaty which
phased out the use of halogenated hydrocarbons (including Freon) due to their proven role in ozone
depletion. In early 1994, the Titan IV program adopted a policy to phase-out the use of Freon for cleaning
of components and simultaneously phase-in the use of a trade product, Bio T-200A.
The Titan IV program had recently finished qualification testing and production of a new component in the
Titan IV A’s SRM TVC system just prior to the buildup of the K-18 launch vehicle. The new component
was the FCV (four places total in the system). K-18 was the first mission with the new FCVs. Prior to K-
18, the valves were an electromechanical valve (EMV), which had been used on the Titan IV and Titan III
program since the early 1970s.
The component design change was driven by the difficulty in manufacturing the EMVs, as well as some of
its internal components becoming obsolete. A cutaway view of the EMV is shown in Figure 37-3. The
intent of the valve redesign was to make as few changes as possible to allow for consistent
manufacturing, part machining, and part procurement, as well as using the same outer housing of the
valve to allow the new FCV to essentially be “dropped in” without any TVC system modifications. There
are several noted differences between the two valve designs. One significant difference is their method
of fastening the packing retainer to the omniseals. Both designs utilize an omniseal to isolate the N2O4
from the valve’s electronic compartment. However, the original EMV design fastened the omniseal and
packing retainer using four screws, whereas the superseding FCV design employed a snap-ring to fasten
the retainer. The four mechanical screws on the EMVs left a much smaller gap (if any) between the
component surfaces, whereas the FCVs snap ring design, while sufficient for its launch duty, had a large
enough gap between the component surfaces that the liquid Bio T-200A cleaning solvent was able to
accumulate on the omniseal spring.
Page 56
NASA/TP-2009-214769
A total of 74 EMVs which had been cleaned with Bio T-200A successfully flew in the Titan IV A TVC
system. The Titan IV program accepted the use of Bio T-200A as the cleaning agent for the new FCVs
by analysis, ground test, and similarity to the cleaning agent’s use on the EMVs. The failure of three of
the four FCVs on this one rocket, compared to the success of 74 EMVs on successfully launched rockets,
suggests that the method of qualifying the FCVs was flawed.
In June of 1994, a subcontractor was commissioned by the FCV manufacturer to test the compatibility of
liquid Bio T-200A with N2O4. Tests showed that N2O4 reacted violently with Bio T-200A after about five
hours of exposure (at ambient pressure and temperature). As a result of this finding, precautions were
added in the EMV (and later the FCV) manufacturing process that included a drying out the components
via a baking process since it was known that a dry film of Bio T-200A is compatible with N2O4. The
o
manufacturer used a 30-minute bake at 100 F, believing that these conditions were sufficient to remove
all the liquid Bio T-200A. However, Bio T-200A has a relatively low vapor pressure and thus requires a
comparatively long time for complete evaporation. Following the K-18 N2O4 spill, an investigation
identified that the FCV manufacturing process indeed left a wet residue of Bio T-200A under the packing
retainer (shown in Figure 37-2).
The presumed sequence of events leading to the subject component failure was as follows:
1. By the completion of the TVC N2O4 propellant tank loading, the cavity adjacent to the packing
retainer filled with liquid N2O4
2. The liquid N2O4 vaporized and migrated into the packing retainer crevice
3. During the tanks’ pressurization, the trapped N2O4 vapor re-condensed into a liquid and reacted
with the residual Bio T-200A in the packing retainer crevice (see Figure 37-4)
o The energy of this reaction damaged the packing seal
o The force of the increased system pressure overcame the remaining, compromised
integrity of the seal components and thus flowed past the seals to atmosphere.
Page 57
NASA/TP-2009-214769
The root cause of this incident was process creep. This was categorized as “Improper GSE/Vehicle
Design” in Appendix C: Detailed Assessment of Incidents. The USAF noted that the process of
accepting the Bio T-200A as a cleaning solvent was not well documented and appears to have been
poorly managed. The chemical compatibility tests (dry film and wet residue) were not given the proper
attention by the manufacturer who believed that a dry film was the only possible remnant of the Bio T-
200A solvent as a result of their baking procedure.
Following the incident, use of Bio T-200A as a cleaning solvent on components in the Titan IV program
was prohibited. All remaining uninstalled FCVs were disassembled, cleaned thoroughly with methyl ethyl
ketone (MEK), cleaned thoroughly with Freon, reassembled, retested, and then returned to logistics. The
reconnaissance satellite was launched successfully on October 23, 1997 on the Titan IV A following the
replacement and retest of the four FCVs.
38 Pad 39B Slope N2O4 Spill (11/6/1997, KSC Pad 39B Slope)
Following the oxidizer (N2O4) loading of Columbia for STS-87 at approximately 9:30 PM on November 6,
1997, a flange gasket (seal) failed in the Pad 39B GSE “cross country” line cross-over valve complex (the
GSE lines that connect the N2O4 farm to the launch pad tower and vehicle). The seal failure resulted in
the spilling of about 25 to 50 gallons of liquid N2O4 onto Pad 39B’s concrete base (proximate cause). The
liquid N2O4 ran down the slope and contacted some live electrical cables and a small electrical fire
followed. The wind direction at the time of the spill was such that the oxidizer cloud drifted away from the
launch pad tower and the vehicle.
During the N2O4 GSE drain down following OV-102’s oxidizer servicing, the cross-over valves were
closed. A water hammer effect from the liquid column in the GSE lines opened a gap at the cross-over
valve flanges (inlet and outlet). This gap enabled liquid oxidizer to leak out onto the launch pad slope (the
concrete incline from ground level to the base of the launch tower).
The SCAPE van driver was the first to detect the spill when he smelled oxidizer in the area of the fuel
farm (see Figure 38-1 for a geographic layout of Pad 39B). Safety officials at the launch pad perimeter
noted the source of the oxidizer vapors was the Pad 39B west slope. Three technicians in SCAPE were
directed to proceed to the general area of the vapor source from their location on launch pad tower 107-
foot level. Upon arrival, the technicians noted that heavy N2O4 (NO2) vapors were leaking from both the
inlet and outlet flanges of the return cross-over valve. They also noted that there were still several
Page 58
NASA/TP-2009-214769
gallons of liquid N2O4 on the ground near the flanges and at the bottom of the slope in a grassy area.
Sparks were seen shortly after the arrival of the technicians; therefore, two of the three evacuated. Some
of the liquid N2O4 that had flowed down the slope incline contacted the electrical conduits routed along
the ground that supplied power to nearby lighting. The technicians were unaware that the N2O4 had
dissolved some of the cabling insulation.
~Wind Direction
Vehicle Location
(Not Shown)
Spill Location
Oxidizer Farm
Fuel Farm
Cross Country Lines
Since it was late at night, the technicians could not see the entire spill area very well. This was a result of
the area being poorly lit during nighttime operations. Therefore, the technicians decided to activate some
additional lighting at the cross-over valve complex. When power was issued to the lights there were
several sparks and a small electrical fire ensued. The lights were quickly turned off and the small fire was
put out with water. During the clean-up effort, one technician inhaled N2O4 (NO2) vapors as a result of his
SCAPE suit zipper not being secured. He was sent to the OHF and released later that night.
There was a concern that the oxidizer vapor cloud had contacted and possibly damaged the Space
Shuttle stack and GSE on Pad 39B. At the time of the spill, the wind direction was away from the vehicle
and launch pad in approximately a southwestern direction (034 to 053 degrees) at a speed of five to ten
knots; however, detailed inspections were still completed. Thermal Protection System (TPS) engineering
performed a vehicle inspection and litmus-pH test to determine if the vehicle TPS was exposed to oxidizer
vapor. They verified no damage or contamination to vehicle TPS. All other engineering groups (vehicle
and GSE) performed inspections and found no damage with the exception of the electrical cables on the
Pad 39B slope.
Page 59
NASA/TP-2009-214769
About two months prior to the spill, an operation was completed on the cross-over valve complex in which
the flange gaskets had to be replaced. This operation was a result of noted leaks at the cross-over valve
complex return valve as a result of (it was assumed) an alignment problem with the valve flanges. The
valve mounts were removed to correct the misalignment issue. The mount removal was completed
without authorizing the out of configuration condition as it was against what was required per the cross-
over valve drawings.
During the repair procedure, the flange gaskets were replaced; however, there was a discrepancy in the
Pad 39B drawing for the flange. An engineering order had been issued in 1978 to the Pad 39A drawings
to require the installation of American National Standards Institute (ANSI) sized gaskets. The Pad 39B
drawings were never corrected; therefore, an undersized gasket was installed. It was also found through
a detailed analysis of the flange gasket installation procedures that the flanges were not leak tested with
GN2 following the 24-hour retorque of the flange bolts. This could have led to a change in the gasket
clamping force. The combination of the undersized gasket and an inherent preload from an inadequate
support structure under the valve complex (improper configuration management) were the root causes of
this incident. The problem report closure statement for the spill stated the following:
“The flange gaskets which leaked were of the white gar lock material and cut to the dimensions
specified per [the Pad 39A cross-country line] drawing. An investigation revealed the material of
the gaskets was compatible and acceptable for oxidizer use. The apparent problem was with the
dimensions specified for 150 lb flange gaskets. The internal diameter was too large, reducing the
sealing surface, and a slight gasket failure led to the leak. A walkdown of the pad B [hypergol
tubing] was performed and all white 150 lb gaskets were identified and replaced with gaskets cut
using the new dimensions released per the [Pad 39B cross-country line] drawing…All subsequent
[helium] leak checks and propellant flows showed no signs of further oxidizer leakage.”
Following this incident, the Pad 39B drawings were updated to require the installation of the proper flange
seals. All the cross-over valves were removed and replaced with flanged pipe as they were no longer
needed for hypergolic propellant loading. This reduced processing time on the valves and greatly
reduced the risk of a leak at the valve flanges. Figure 38-2 is a photograph of the location where the
cross-over valve complex was previously located at Pad 39B and Figure 38-3 is a photograph of the
location of the Pad 39B fuel cross-over valve complex with the fuel farm and Vehicle Assembly Building in
the background.
Supply Line
Oxidizer Flanges
(Previous Location of
Return Line
Cross-Over Valves)
Figure 38-2: Former Site of Oxidizer Cross-Over Valve Complex at Pad 39B
Page 60
NASA/TP-2009-214769
Fuel Supply
Fuel Farm Line Flanges
Cross-County Lines
Figure 38-3: Former Site of Fuel Cross-Over Valve Complex at Pad 39A
The vapor levels were checked in all of the applicable areas and all were acceptable for ferry flight
closeouts (OV-102 was leaving for Palmdale, California soon after the spill). The root causes of the
hydrazine spill were an insufficient GSE design in OPF3 and lack of experience by the technician that left
the flexhose uncapped (improper training). This spill was the driver for implementation of new permanent
GSE in each Space Shuttle OPF. The new panels were installed, calibrated, and validated in each OPF.
They provide versatility as well as the additional safety of a permanent installation. The installation of
these new panels substantially reduced the possibility of another APU N2H4 fuel spill during this
procedure.
References: Gehman (CAIB Volume I finding F4.2-7 and Volume II pg 427); 1999-232-00007
Page 61
NASA/TP-2009-214769
Following the completion of the hotfire test near the end of the work day, the test team and customer
agreed to delay the subsequent propellant sampling and decontamination until the next day (Friday). The
following day it was discovered that only one person was available for sampling and the rest were out on
vacation. The sampling was completed late in the day, which did not allow enough time for the
decontamination procedure to take place until the following Monday.
On Monday August 7, 2000, the hardware was configured to support the decontamination procedure in
which water was flushed though the propellant lines (both MMH and N2O4). It is usually not good practice
to flush an N2O4 system with water as it forms nitric and nitrous acids. The procedure was performed
using a TPS (Test Preparation Sheet used primarily for non-routine work) rather than a WJI (WSTF Job
Instruction used primarily for repetitive, routine work). The test conductor (TC) proceeded to perform a
vent of the oxidizer manifold even though this work step was not in the procedure or even necessary
since the decontamination was intended to be completed with the manifold pressurized. A large NO2
cloud was then noted at the test cell drain valve. The TC decided to not vent to MMH manifold since it
would possibly mix residual propellants; therefore, he proceeded to cycle the engine valves. Following
this, another “large oxidizer [NO2] cloud was observed and a large combustion event occurred,” as quoted
from the WSTF report. It is assumed that the damage from this event was unnoticeable since the TC
continued on after briefly stopping the test.
Regardless of the previous off-nominal events, the test team decided to proceed with the water
decontamination. There was also supposed to be a nitrogen purge of the system, but the test team
decided to not perform this operation. Deionized water at approximately 170 psig was introduced into the
fuel lines. Shortly thereafter, the technician connected the water from the same source into the N2O4
propellant lines. The technician then proceeded to exit the test cell and as he did, there was another
explosion. A loud bang was heard by the technician and water was observed flowing out of an unknown
source from the test article. The test system was then vented remotely. The following items had either
failed or were damaged:
The proximate cause for the failure was inconclusive. The most likely scenario is that MMH and N2O4
reacted on the oxidizer side of the engine valve (a bi-propellant valve) possibly through a Kalrez seal
failure. The engine valve has redundant seals on both the oxidizer and fuel sides. In between the seals
there is a small cavity with a check valve that is designed to vent propellant seepage between the seals.
The reaction either propagated a shock wave into the water filled system or accelerated columns of liquid
in the water and vapor filled oxidizer system. Another possible proximate cause for the failure is the
removal of the vacuum conditions before the completion of the decontamination process.
Working steps out of sequence, not performing steps, and completing the decontamination procedure
several days later than required (operational human error) along with performing the procedure with a
TPS rather than a WJI were the root causes of this incident. Proper care was not taken with respect to
procedure adherence and development.
Reference: “Investigation Report of the WSTF Peacekeeper Axial Engine Valve Damage”
Page 62
NASA/TP-2009-214769
The proximate cause of the fuel ignition is unknown. Earlier, during the oxidizer system decontamination
(see Section 40), an unexplained rupture of an oxidizer feedline had occurred on the test article. It is
thought that residual oxidizer could have caused the MMH ignition, but it is uncertain. Other possibilities
include rust from the test stand or static electricity as proximate causes. The root cause of this incident
was operational human error as a result of the lack of oversight and careful planning following the incident
on the same test article a few days earlier.
The quick reaction (albeit unsafe since he was not wearing the proper personal protective equipment) of
the safety observer, likely minimized the hardware damage and prevented injury. According to WSTF
Standard Instruction (WSI) PROP-0049, “The Buddy System in the Propulsion Department,” the third
technician should have been attired in ILC.
Reference: Gorham
The IPOV was in the process of being validated following some modifications that were made to the
valve. The operating pressure at the valve inlet was approximately 276 psia. The test was simply flowing
liquid N2O4 from one tank into another by cycling the valve. The pressure transducer was in the system
to monitor this inlet pressure. Approximately 36,000 (of the planned 80,000) cycles of the valve, with a
flow duration of approximately 80 ms and a maximum flowrate of 1.91 lb/sec, had been completed when
the explosion occurred. The proper emergency procedures were immediately implemented following the
incident. These procedures included securing the test cell and deactivating the air handler for both the
test cell and the attached building. There was no reported fire following the explosion.
Initial inspections of the pressure transducer indicated that the transducer failed as a result of one of the
following: over-pressurization, inadequate design, or cyclic fatigue at a highly stressed location. Since the
pressure transducer failed, data from it at around the time of the failure may be suspect; therefore, the
investigation included a detailed inspection of the nearby system tubing to determine if there was an over-
pressurization event. Close inspection of the pressure transducer itself would indicate if the transducer
failed from cyclic fatigue or was subject to a ductile failure from a single overload event.
Page 63
NASA/TP-2009-214769
Approx. Manifold
Location Approx. Pressure
Transducer Location
IPOV Test
Article
It was found that the pressure transducer failed at welded connection between the port (media) housing
and the sensor housing. Figure 42-2 is a photograph of the pressure transducer following the explosion.
It was calculated that the approximate pressure to fail this butt weld connecting the two 300 series
stainless steel components was approximately 12,700 psi. It was also found that the system tubing
showed no sign of yielding due to an increased pressure; therefore, the system pressure did not exceed
the calculated tubing yield value of 3,400 psi. The pressure transducer diaphragm was found to have
failed at an approximate calculated pressure of 4,000 psi. The pressure then built up very quickly in the
small volume around the diaphragm and was unable to properly relieve through the 0.157-inch diameter
port at the top of the transducer. This allowed the pressure in this small volume to roughly reach the
12,700 psi transducer burst pressure.
The component failure (over-pressurization) was a result of a fatigue fracture (crack) of the pressure
transducer diaphragm (0.002 inches thick). This allowed silicone oil to leak into the small volume
surrounding the diaphragm. Approximately 0.8 mL of silicone oil was used to fill the volume between the
diaphragm and the silicon sensor. The liquid N2O4 reacted with the silicone oil to form nitrated phenols,
including dinitrophenol and trinitrophenol (picric acid), which are shock-sensitive compounds. Some of
the nitrated phenols remained between the diaphragm and the sensor, while the rest were diluted into the
liquid N2O4 in the test system. When the IPOV was subsequently cycled, it is assumed that the shock
wave in the liquid detonated the shock sensitive compounds in between the diaphragm and the sensor.
N2O4 (NO2) vapors were then released from the openings in the system. The vapor cloud filled the test
cell and was carried by the wind through an air handler of a nearby building.
The presence of the silicone oil was not mentioned in the pressure transducer specification from the
manufacturer. This is the case for many pressure transducers of this specific design. Close scrutiny of
specifications during hypergol component procurement is extremely important. This incident was difficult
to prevent since the WSTF personnel did not know that there was silicone oil in the pressure transducer,
but an improper GSE design (component procurement) is designated as the root cause. The wind
corridor was also assessed and, as a result of this incident, it was altered to reduce the risk to the
surrounding buildings from an oxidizer vapor release.
Page 64
NASA/TP-2009-214769
Reference: Anderson
The engineers had noticed a decrease in the oxidizer pump flow rate (34-50 gpm versus 200 gpm) during
the 18,600 gallon loading of the Stage I N2O4 tanks. A meeting was held to discuss this reduction in flow
rate in which the contractor and government team chose to continue on with the Stage II load. The
consensus of the meeting was that an internal filter had become partially clogged at the oxidizer farm.
The personnel had seen this sort of occurrence previously. Operational human error is considered a root
cause as a result of the loading team not investigating the decreased flow rate issue further.
Following the pump explosion, the loading team quickly removed power from the pump, closed the flow
control valves at the oxidizer farm, and immediately evacuated the pad. The large NO2 cloud that was
created following the explosion was found to pose no danger to the local communities; however, a 5,000-
or 12,000-foot clear (depending on the wind speed and direction) was put in place around the oxidizer
farm.
Page 65
NASA/TP-2009-214769
N2O4 Farm
Motor
Pump
The incident failure analysis concluded, according to Crass, that there was a “cooling anomaly at the
[carbon graphite journal] bearings…leading to [an explosion] in the stator can winding area.” It was found
that the most likely contributor to this was a clogged filter in the pump recirculation loop that circulates
liquid through the rotor cavity to cool the front and rear bearings as illustrated in Figure 43-3. Data
analysis showed that the upstream and downstream temperature transducers indicated an increase in
o
temperature of 8 F across the pump. The low flow rate of 40 gpm and high ambient temperatures (high
o
of 86 F for 8/12/2003) accelerated the failure of the pump. At these flow conditions, the pump was not
Page 66
NASA/TP-2009-214769
able to be properly lubricated as a result of the high vapor pressure of liquid N2O4 (these particular pumps
are designed to be lubricated with the liquid that they are pumping). The excessive bearing wear led to
damage to the stator liner can, allowing oxidizer to leak onto the hot copper windings. The N2O4 reacted
with the copper winding insulation and caused the coil wire to short, which was the proximate cause of
the explosion.
Following the explosion, there were some design changes made to the pumps since an improper pump
design was a root cause, these changes include the following:
Machining of the pump from a single billet of stainless steel rather than casting them
Serrations were added on the impellor housing and rear bearing housing for improved sealing
capability
The addition of a thermowell for thermocouple monitoring rather than the old thermoswitch which
proved to be unreliable
This incident encouraged the Space Shuttle Program to consider alternate pump designs for its
hypergolic propellant loading system. The centrifugal pumps at Pads 39A and 39B used for Space
Shuttle hypergol loading were replaced with centrifugal pumps with the design changes listed previously;
however, the changes did not completely mitigate the failure mode that caused the LC-40 pump to
explode. This risk was accepted by the program and engineering team; although, it was mitigated by
implementing the following controls:
All pumps were removed and replaced following the LC-40 pump explosion
The pumps were leak checked using a helium mass spectrometer
A programmable logic controller was installed to deactivate the pump when the temperature
o
reached 140 F along with the normal health monitoring that has always been installed on the
pumps
Page 67
NASA/TP-2009-214769
Run-times and start/stop times were tabulated to establish a baseline for a preventative
maintenance program (removal and replacement is required once every 100 hours for oxidizer
and every 150 hours for fuel)
Vibration analysis monitoring devices are installed on the pumps each time they are used
o The data is evaluated and released in a formal report following the load
There have been four other similar occurrences (three oxidizer pumps in 1963, 1964, and 1980 along with
one fuel pump in 1962) throughout the history of the Titan program. The three failures in the 1960s were
a result of incompatible bearing material, which was resolved, and the failure in 1980 was a result of a
pin-hole leak in the stator liner can, which was mitigated by implementing a helium mass specification
leak check.
44 HMF RP01 N2O4 Spill (6/5/2004, KSC HMF M7-961 East Test Cell)
On June 5, 2004, at the HMF M7-961 East Test Cell, a technician was exposed to a few drops of liquid
N2O4. The drops ran down the technician’s glove and onto his left arm while attempting to reinstall a flight
cap onto MD306 (RRCS oxidizer tank entry sump bleed) on RP01. The technician rinsed off his arm for
about 20 minutes in a safety shower and was then sent to a local hospital. He was released later that
day. There was not any reported hardware damage. The procedure included pre-operations for later
testing of the RRCS helium system isolation valves, regulators, check valves, relief valves, and burst
disks.
The flight cap removal procedure began with the technician completing the bleed between the flight cap
3
and the AHC (0.0697 in for a ¼-inch AHC and flight cap assembly), as the procedure stated. This
process had the technician install a Tygon tube to the bleed port on the flight cap and run the tube into a
nearby bucket of water. The two reasons behind the flight cap/AHC interstitial volume bleed are to vent
off the small volume from any N2O4 (NO2) vapors that collect as a result of in-specification leakage of the
poppet and to verify that the AHC poppet is not stuck in the open position. The two technicians working
the operation were attired in the required PPE (hypergol compatible gloves, apron, chemical goggles, and
face shield since the work was overhead). Supplied air respirators were on stand-by in case they were
needed for any reason.
When the bleed port was opened, the technician noted that he saw roughly the expected amount of
oxidizer bubbles in the bucket of water. The two technicians monitored the bucket for about a minute at
which time they reported that the bubbles had stopped.
The technician then proceeded to remove the flight cap from the AHC (the proximate cause of the spill) at
which time he noted that there was a slow liquid leak originating at the AHC poppet. The two technicians
immediately upgraded their PPE to supplied air respirators and began to contain the leak with an active
aspirator hose. The removed flight cap was not able to be reinstalled as a result of its degraded
condition. The technicians requested a new cap be installed, which was subsequently brought into the
test cell by the lead technician in street clothes. An attempt was then made to install the new cap without
using the installation tool since this would require the technician to be located beneath the leaking AHC.
It is claimed in the incident report that when this was attempted, the leak rate increased, as a result of a
hydraulic lock from the presence of liquid between the AHC poppet and the flight cap, which had a closed
bleed port. This does not seem to be realistic. The benefits of using the tool (the technician could install
the cap much quicker and at an increased distance from the cap, reducing the likelihood of injury) are
likely to out-weigh the risk, as long as the task was completed carefully.
On a later attempt to install the flight cap, liquid N2O4 ran down the glove of one of the technicians and
onto his arm. It is believed that his skin was exposed to about two or three drops of oxidizer. He
immediately located an eyewash station to run water over his arm for several minutes. Meanwhile, the
other two technicians (one in PPE and one in street clothes) configured the aspirator hose using an
Page 68
NASA/TP-2009-214769
adhesive tape to contain any liquid that was released from the AHC. The three technicians then left the
test cell and the exposed technician was sent to the hospital from which he was released back to work
about 90 minutes later.
SCAPE personnel arrived at the scene about four hours after the initial spill. They commenced with a
procedure to connect a GHC to AHC MD306. Once the GHC was connected and the test cell was
verified to have a 0.0 ppm breathing zone, the cell was opened for controlled work.
The RRCS oxidizer tank was later drained for the AHC poppet replacement procedure. This operation
was successfully completed. Following the spill, it was determined that roughly 1.4 gallons of N2O4
leaked through the MD306 poppet. The majority of this was contained using the aspirator.
The incident report claims that the exposure may have been prevented if the procedure had been written
more clearly and the technicians had been more knowledgeable of the chemistry that occurs when an
N2O4 system is vented (both possible root causes of the spill). The procedure directed the technician to
vent (bleed) the volume between the AHC poppet and the flight cap for 15 seconds. The incident report
goes on to say that the procedure did not specifically state that the volume should be vented for “no more
than” 15 seconds; therefore, the technicians allowed the oxidizer in the interstitial volume to condense
back into a liquid as a result of the dropping pressure and temperature during the vent. This theory is
anecdotal and cannot be supported with any experimental or operational data. It was calculated that the
leak rate of the N2O4 from the AHC was approximately 22.7 cc/min of liquid. This amount of liquid would
have been clearly evident in the bucket of water by a trained technician even though it was later noted
that the piece of Tygon tubing that was used for the operation was old (opaque).
The course of action taken following the initial spill (containing the spill as best as possible and then
standing down until a SCAPE technician could continue the work) was likely the best possible solution at
the time. The effort made by the technicians to reinstall the flight cap, thus containing the leaking AHC
poppet, were noble; however, they put themselves at risk by installing the cap without the proper tool and
completing the task without the appropriate PPE.
The most probable cause indicated in the incident report was likely not the reason for the technicians not
seeing any liquid N2O4 exiting the Tygon tube. There is not a sufficient amount of data supporting or
disputing the most probable cause presented in the report. Another possible scenario brought forth by
NASA systems engineering was that the flight cap bleed port was blocked (partially or fully) by nitrate
crystals that form when N2O4 (or NO2) within an iron or iron alloy containment vessel is given the chance
to mix with humid air and then later dry up. Proper training of the engineers and technicians was a
possible root cause of this incident; however, it remains unclear what the exact mechanism was behind
the spill. The flight cap was tested following the incident and it was found that GN2 was able to flow freely
through the bleed port; however, nitrates could still have been present. They could have either dissolved
in the liquid N2O4 or were not large enough to constrict the GN2 flow.
Improvements that were implemented following this spill include using a new piece of Tygon tubing for
every flight cap bleed, inserting a loop in the tubing, and creating a consistent flight cap removal operation
between the HMF, OPF, and pads.
45 WSTF N2H4 Spill Following Manual Valve Failure (9/30/2005, WSTF TC 844B)
On September 30, 2005, at WSTF TC 844B during a low pressure test, a ½-inch manual valve bonnet
(handle) failed catastrophically. The failure caused the valve bonnet and stem to be ejected from the
assembly, thus spilling approximately 74 gallons of liquid N2H4 into the test cell (proximate cause of the
N2H4 spill). Most of the liquid N2H4 was captured in the floor drains which were connected to a 1,500
gallon secondary containment tank; however, some was lost as vapors into the air. There were no
reported injuries. Figure 45-1 is a photograph of the lower bonnet fracture surface.
Page 69
NASA/TP-2009-214769
The failure began as an intergranular crack in the nickel-plated, aluminum bronze bonnet threads
(fracture surface shown in Figure 45-1). It is believed that the crack began as a result of the ethylene
propylene rubber and Teflon soft seals allowing corrosive agents to slowly leak into the thread region.
Figure 45-2 shows a cross-section cutaway of the manual valve. The Teflon soft seals can be seen
clearly near the middle of the threaded shaft. The corrosive agents reacted with ammonia, which came
from the decomposition of N2H4 (N2H4 1/3 N2 + 4/3 NH3). This accelerated the local corrosion and
cracking. This was the first failure for this type of valve since they were procured and installed in several
hypergolic systems at WSTF in 1994.
The manual valve was used for the isolation of a leak-check leg of the “New Gas Generator Valve
Module” subsystem in the Improved Auxiliary Power Unit (IAPU) system. This system is used to study
long-term exposure of the IAPU to hydrazine under simulated Space Shuttle conditions.
The valve had been cycled roughly 380 times and seen approximately 177 pressure cycles at around 400
psia (the last 42 cycles were in 2005) since its installation in September of 1994. In 2005 testing began
on the IAPU system using high-purity N2H4 (rather than monopropellant grade N2H4) to determine if there
were any significant differences in the corresponding catalyst bed temperatures. High-purity hydrazine is
defined as 99.5% pure N2H4. Monopropellant grade N2H4 has a purity of 98.3%. During the testing, an
internal leak was discovered in the system. The workaround of this internal leak was to alter the system
valve configuration. The new configuration had the soon-to-be failed ½-inch manual valve in-line with the
main catch tank and the bypass catch tank which resulted in a pressure of 50-psia on the manual valve
that subsequently failed. While in this static configuration, the manual valve bonnet fractured and was
ejected from the valve into the test cell, spilling the N2H4. The altering of the nominal configuration could
be considered a root cause, along with the improper design (primarily material incompatibility) of the
manual valve, itself.
Immediately following the incident, the Automatic Dialing Alarm System (ADAS) began to call the
responsible engineers and the WSTF fire department. The test cell was not being monitored by
personnel at the time of the incident. The ADAS had detected a significant pressure drop over a short
period of time in the test cell. No one was able to be reached with the automated system until the final
call in the call sequence was made to the WSTF fire department. This call was made 16 minutes after the
incident occurred.
Page 70
NASA/TP-2009-214769
Fracture Location
The response of the personnel following the incident could have been better. However, proper steps
were taken to evacuate all non-essential personnel from the area and activating the TC 844B exhaust
fans about one hour after the incident following an assessment of the wind direction. Several different
personnel noted that they saw smoke or vapors exiting the test cell underneath doors and out the exhaust
vents. It was also noted that the vapors in the test cell were so dense that local cameras were unable to
obtain a clear image. The water deluge system was only activated for three minutes (at a flow rate of 150
gpm) in the test cell (in attempt to dilute the N2H4) to reduce the risk of damage to the test equipment and
the possibility of a fire. This added approximately 450 gallons of water to the secondary containment
tank. There was no evidence of a fire or explosion during a post-incident inspection. It would not have
taken much for an ignition with the reported density of N2H4 vapors in the test cell prior to the activation of
o
the water deluge system. The mean temperature on Friday, September 30, 2005 was 72 F. If it had
been warmer, the outcome of this incident could have been much worse.
Page 71
NASA/TP-2009-214769
Test cell 844 has been exclusively an APU component test bed since 1994; therefore, it was designed for
fuel only and there were no considerations for oxidizer spills. The secondary containment tank and the
water deluge system were part of the overall system/facility design. Those portions of the system worked
as they were intended once the decision was finally made to activate the water deluge. The containment
tank was sampled and the water/N2H4 mixture was pumped into barrels, and sent off-site to an incinerator
for disposal. There was no cleaning completed on the secondary containment tank following the N2H4
spill.
The incident report stated that the following items are contributing factors to the manual valve bonnet
failure by environmentally assisted cracking with the final two primary bullets being the root causes
(improper GSE component design):
The incident report stated that the following were not contributing factors to the failure:
46 STS-121 FRC3 N2O4 Spill (1/9/2006, KSC HMF M7-1212 West Test Cell)
On January 9, 2006, an N2O4 spill occurred (approximately 2.9 gallons) in the HMF M7-1212 West Test
Cell on the 10-foot level. This occurred when the FRC3 AHC poppet assembly for MD122 (FRCS
oxidizer propellant tank lower compartment channel bleed) was being removed and replaced (proximate
cause). Two technicians were evacuated from the building and sent to the OHF following the scent of
N2O4 in their SCAPE suits. Two other involved technicians were sent to the OHF as a precaution. It was
determined that all four had no injuries. It was also determined that the SCAPE suits had not failed (e.g.
they did not have any holes in them). There was notable damage to flight hardware and ground support
equipment in the test cell. Several PRs were opened during the cleanup process on the following
systems: TPS, TCS, STR, OHE, ORH, and OEL/INS.
A few opinions were brought forth as to the possible cause of the spill. It is unknown if one single event
caused the liquid to fill the MD122 line and spill into the M7-1212 west test cell or if it was a combination
of several different events. Operational human error was designated as the root cause of this incident as
a result the engineering team incorrectly predicting the response of the vehicle system during the removal
and replacement procedure.
Page 72
NASA/TP-2009-214769
With the vehicle in a horizontal orientation, the MD122 line runs from the high point of the oxidizer tank
lower compartment channel, through an interface at the tank outlet flange, and out to an AHC located in
an externally accessible door on the FRCS outer mold line (AP28-00 door). It is known through pressure
measurements that the MD122 and MD120 connected GSE lines were clear of any liquid immediately
following the two, five-minute purges; however, it is not known if the propellant acquisition system channel
in the lower compartment was broken down (i.e. it did not contain any liquid). If the screen was not
completely broken down, it is possible that liquid oxidizer “wicked” up the screen following the purges to
the point where MD122 connects to the acquisition screens. See Figure 46-1 for visual aid of the
propellant acquisition system screens and the MD122 line. The absence of liquid in the lines can be
confirmed by noting that there was not any significant pressure increase in the lines. The tank was not in
o
thermal equilibrium. The tank shell (skin) temperature rose from 49 F, when tank venting was complete,
o
to 58 F, when MD124 GSE vent valve was opened for the emergency vent through MD124 in the upper
o
compartment of the tank. When the two purges were complete the tank shell temperature was 54 F.
o
From the time of the spill to the emergency MD124 vent, the tank shell temperature rose from 57 F to 58
o
F. The fluid and gas temperatures likely rose by a degree or two during this time.
Screens
Figure 46-1: Solid Model of RCS Propellant Tank in (Roughly) the Horizontal Orientation
At the time of the spill there were three pressure transducers online for use. All three pressure
transducers indicated no change until just after 12:29 PM when the oxidizer tank out pressure toggled
down 1.6 psi from 14.4 psia to 12.8 psia. The venting of the tank through MD124 took place at 12:27:24
PM. This, however, does not mean that the tank pressure was not in flux. As a result of the coarse data
resolution, the oxidizer tank out pressure could have been changing within a range of +/- 1.6 psi.
Prior to the spill, it was determined through x-ray analysis that the N2O4 propellant tank contained 19.5
gallons of N2O4. Following the spill, it was determined that 2.9 gallons of N2O4 had been lost through the
venting process and the spill itself.
In the hours and days following the spill many problems were reported as a result of the N2O4 (NO2)
vapors reacting with different materials on the Space Shuttle forward module. A photograph of the
module following the spill is shown in Figure 46-2. Damage to the Koropon structural coating can be
seen along with a couple of missing thermal tiles. A total of ten thermal tiles (highlighted in yellow in
Figure 46-2) were later removed and replaced before the perimeter of the spill was bounded. Other PRs
were taken to assess issues with wiring and Neoprene adhesive wraps that are used to attach wires to
Page 73
NASA/TP-2009-214769
tubing and are known to react with N2O4 (NO2) vapors and cause corrosion on stainless steel tubing
through chloride attack.
MD122 AHC
The spilled oxidizer appears to have reacted with Koropon, RTV, and silver plating (nutplates and barrel
nut elements). The oxidizer vapors reacted with the Koropon and removed a large area, as shown in
Figure 46-3, and passed through the nutplate holes and around the perimeter of the phenolic QD service
panel into the interior of the FRCS module, as shown in Figure 46-4, causing damage to the Koropon on
the interior side of the FRCS module.
Area of Missing
Koropon
Figure 46-3: View Looking at R/H Side Down-Firing Thrusters (F2D & F4D)
Page 74
NASA/TP-2009-214769
The spill also caused damage to a few thermal blankets located within the FRCS module. The primary
visible effect on TCS blankets when exposed to N2O4 (NO2) vapors is the degradation of the thin metallic
foil coating. Depending upon the amount of exposure, this effect can be delayed. The blankets were
evaluated two days after the spill occurred, and three blankets were identified as contaminated. These
blankets were removed. The remaining blankets were evaluated later with further degradation noted.
A report was put together by HMF USA OMS/RCS engineering management following the input from a
four-panel-member board, which included one NASA representative. The purpose of the report was to
summarize the spill event and form recommendations to improve processes in the future. Two findings
and five observations were noted in the report. A summary of the findings/observations and
recommendations are tabulated in Table 46-1. Following the table are several observations put forth by
NASA OMS/RCS engineering.
Finding/Observation Recommendation
Evaluate and develop alternative methods to reduce
Procedure was reviewed and alternative purge
risk of liquid release. Evaluate process for
processes were discussed in house by NASA/USA
independent review of complex hazardous tasks
engineering system experts.
using the First Article Review Process.
Failure to develop contingency plans in the event Include Emergency Procedures for unexpected liquid
of unexpected liquid release. release.
Personnel reported smell of oxidizer in SCAPE
Brief individuals involved in the SCAPE operation on
suit and reported to the test team. Each member
the proper response to possible hypergolic
of the test team directed the potentially exposed
contamination inside the SCAPE suit. Re-emphasize
employee to leave the work area. Employee
training in removing individual from contaminated
egressed after the third request to leave the test
area, (squat and squeeze), report anomaly, depart
cell. An employee in the relief SCAPE crew stated
area, perform individual change out, and maintain the
later that he smelled oxidizer in his SCAPE suit,
buddy system.
but failed to report it to the test team.
Page 75
NASA/TP-2009-214769
Finding/Observation Recommendation
Multiple occurrences of SCAPE discipline
Management to brief individuals involved in the
breakdown; breathing air was not maintained
SCAPE operation on the importance of maintaining
either by air bottle or hardline and communication
positive breathing air and communications at all
between SCAPE personnel and test team was not
times.
maintained.
Test Team evaluated the hazardous conditions
and determined the safest course of action was to
complete the entire operation, including the clean-
Conduct Post Test review and evaluate test team’s
up of released oxidizer prior to mishap evaluation.
actions for compliance of a USA operating procedure
Unexpected oxidizer release was reported to
related to Reporting, Investigating, and Resolving
Safety Console and HMF Management. Witness
Mishaps.
statements and data collection commenced
without delay, and were completed by the end of
shift.
Test team performed an engineering risk
assessment during the development of the outline
Review First Article TOP development process.
and table top reviews however failed to document
per the SFOC Risk Assessment process.
Spill Protection did not adequately contain quantity Evaluate Spill protection process and develop desk
of defined credible spill and failed to protect instructions to ensure proper level of spill protection
adjacent hardware from oxidizer release. corresponds to reasonable risk of release.
Page 76
NASA/TP-2009-214769
Since the spill occurred, a very similar procedure was successfully completed on forward module FRC3
again. Engineers and technicians completed the procedure review and operation with the lessons
learned from the 2006 FRC3 spill in mind.
47 Conclusion
Many hypergolic related incidents have been discussed. A quick summary of the incidents mentioned
previously can be seen in Appendix B: Summary of Incidents. Some common lessons learned deduced
from the various root causes are shown in the following list. If these items were properly addressed prior
to the incidents, prevention may have been possible (in hindsight) or the impact or consequence of the
incident could have been reduced.
Improper configuration control and internal or external human performance shaping factors can
lead to being falsely comfortable with a system
o Vent systems are often neglected and treated as non-hazardous even though they can
capture and contain hypergolic liquids (especially in low points)
o Aging support hardware should be routinely inspected to reduce the risk of a failure
during critical operations
Communication breakdown can escalate an incident to a level where injuries occur or hardware is
damaged
Improper propulsion system and ground support system designs can destine a system for failure
o Low points in GSE should be designed out
Improper training of technicians, engineers, and safety personnel can put lives in danger
o Inadequate knowledge of electrostatic discharge while working fuel operations can lead
to a fire or explosion
o Knowledge of transducer offsets is very important for system oversight
o Unknown incompatibilities (from lack of training or research) with propellants can cause
surprising failures
o If an incident does occur, the system should immediately be placed into a stable
configuration; following this, the procedure should be stopped to assess the
problem and its possible ramifications
Page 77
NASA/TP-2009-214769
Table 47-1 and Figure 47-1 summarize the fuel and oxidizer incidents. Note that if the numbers in the
table are totaled, they do not sum to the total amount of fuel and/or oxidizer spills summarized in this
document. This is a result of some incidents having injuries and a fire or multiple root causes, for
example. A larger, more extensive list of the data shown in Table 47-1 can be seen in Appendix C:
Detailed Assessment of Incidents.
Page 78
NASA/TP-2009-214769
Some lessons learned from the Apollo program related to hypergol loading equipment according to J.
Tribe include:
On-board vehicle instrumentation was limited to that required for flight evaluation; this dictated
use of GSE instrumentation to monitor critical vehicle parameters during ground operations-a less
than desirable configuration
Operational visibility from the control room was minimal and there was extensive reliance on
technicians (usually in SCAPE) to read gages correctly and position multiple manual valves
o The servicing and test disconnects on the command and service module (CSM) were
challenging to manually position correctly
o This inevitably led to a mis-configuration on Apollo 16 that resulted in a CM RCS tank
ruptured bladder, roll back of the Saturn V stack and de-stack of the CSM and lunar
module; lack of flight system instrumentation was a direct contributor to this event
As the design matured, configuration changes increased complexity for ground operations and
greatly extended servicing timelines
o SM RCS changes from individual block 1 “quads” to block 2 to the propellant storage
module installation are an example (8 tanks to 16 tanks to 25 tanks)
o Each “quad” was a stand-alone system until the propellant storage module
interconnected them
Multiple individual tanks on CSM required multiple access panels and disconnects
o Compare: # of tanks # of servicing panels # of disconnects
CSM Skylab 31 22 56
Shuttle OMS/ARCS 8 2 22
o This multiplicity resulted in a complex ground servicing operation with an extensive fluid
distribution system, valve boxes, bleed units, ullage cylinders, purge panels and the need
for a weather-protected 360 degree access
Page 79
NASA/TP-2009-214769
Apollo servicing operations required a variety of loading methods – mostly sequential and time-
consuming
o SPS fill used on-board gauging/totalizer to determine flight loads
o RCS fill used a combination of evacuated tanks with load by weight and fill to overflow,
removal of a specified ullage and a manual PV determination to confirm flight loads
o Flight loads were accurate but the methods were time-consuming
During the life of the program, significant propellant spills (especially during Apollo 7 preps) drove
increased need for spill collection and containment
o The installation of improved scuppers at vehicle interfaces and facility precautions were
complex and time-consuming
Different fuels for different flight systems (A-50 and MMH) doubled the number of servicing units
and fluid distribution systems
The Space Shuttle Program corrected many of these deficiencies with
o Remotely controlled operations from the LCC
o Extensive use of software loading programs
o Remotely operated disconnects and leak checks
o Gang servicing valve disconnects for OMS and ARCS
Some type of human error can be traced to nearly every studied incident as a root cause, whether it be
an error in the design phase or an error prior to or during operational use of hardware containing
hypergols. Humans are most definitely not perfect and even when the most knowledgeable personnel
are intimately involved in the design phase (vehicle or GSE) or during an operation, mistakes can be
made and critical items can be overlooked. One can deduce, however, that most incidents happen during
some sort of dynamic operation. Hypergols tend to be very stable in a static configuration (as long as the
compatibility characteristics have been well addressed). At the end of the day, an appropriate quote
related to the majority of hypergolic spills and fires comes from the 9/11 Commission Report, “The most
important failure was one of imagination.”
Advance warning (prior to any liquid or vapor release) was available in several of the incidents to the
technicians in the vicinity of the spill and/or the engineers that were monitoring from a remote location.
The warning indications include off-nominal data (remote or local), off-nominal system characteristics,
and/or local changes that occurred without human intervention. Some of these went unnoticed or were
ignored during the operation, thus resulting in an incident. There was advance warning in 19 out of 38
total incidents (50% of the time). This percentage does not include spilled fuel as an advance warning of
a fire (5 occurrences). Depending on the local environment, there is a reasonable probability that if
hydrazine (or one of its derivatives) is spilled, there will be a fire; therefore, the fuel spill itself is an
advance warning of a fuel fire. Roughly 42% of the fuel fires documented in this report resulted in a fire
or explosion. The Titan IV K-9 N2O4 spill likely had an advance warning; however, there was no one in
the vicinity of the indications to receive the warning, therefore, this was not included in the above
percentage along with the OV-101 APU spill in 1977, since it was unable to be determined if there was an
advanced warning for this incident. Located in Appendix B: Summary of Incidents is a summary of the
advance warnings for each incident (if there was one).
Hypergolic rocket propellants have proven to be a highly reliable asset in manned and unmanned space
flight; however, their maintenance on the ground has proven to be relatively difficult. Do the operational
risks from possible human errors or hardware failures causing a catastrophic incident outweigh the
usefulness of hypergols even though they have been used for the last 50 years of manned and
unmanned spaceflight? One would have to say probably not, since hypergols are so widely used in the
space industry currently and are being proposed to be used on many vehicles in the future. Therefore,
ground operations on hypergol systems have become increasingly scrutinized for possible unknowns,
and rightfully so. The data shown in this report are not an example of why we should not be using
hypergolic propellants on spacecraft and launch vehicles, but rather what we can and should do to
mitigate possible unforeseen ground operation and/or design problems.
Page 80
NASA/TP-2009-214769
48 References
Anderson, John C. “Test Cell 831 Druck Pressure Transducer Failure on March 25, 2003.” Johnson
Space Center, White Sands Test Facility Laboratories Office. Investigative Report Number
WSTF-IR-0185-001-03. May 5, 2003.
Applewhite, John B. “Update on L3D Removal Incident.” Informal email. May 31, 1995.
Baum, B. “Impact Sensitivity of Chem Pump Stator Materials After Oxidizer Exposure.” Martin Marietta
Corporation Inter-Department Communication 0483/81-32. January 28, 1981.
Benson, Charles D. and William Barnaby Faherty. “Moonport: A History of Apollo Launch Facilities and
Operations.” NASA Special Publication-4204. 1978.
Bowman, H.D. “Oxidizer Leak/OPF High Bay 2, 17 Feb. 84 at 0945.” Interdepartmental Communication.
2301-84-021HDB. February 18, 1984.
Buzzard, Frank et. Al. “NASA Management Briefing of OV-105 R1A Thruster Fire Mishap.” Presentation
slides given by NASA Investigation Board. May 18, 1995.
Comfort, James L. “ORB Minutes – SLC-40 Oxidizer Farm Pump Incident.” Memo for 45 OG/CC,
Department of the Air Force, Patrick AFB, FL. August 14, 2003.
Cotton, R. K. “Pad A Monomethylhydrazine (MMH) Fuel Farm Mishap.” Lockheed Space Operations
Company Mishap Investigation Board Investigation Report. January 21, 1986.
Craig, Anna F. “Procedure for Connecting, Disconnecting and Maintaining the Cleanliness of Forward
Reaction Control Subsystem and Aft Propulsion Subsystem Disconnect Couplings.” The Boeing
Company Specification. ML0310-0032 Rev. G. February 21, 2002.
Crass, Larry, Art Geer, Steve Cronau, and Mark Maccan. “SLC-4E Blast Cage Modifications/Ox Pump
F/A Results.” Baker Engineering & Risk Consultants. ERB 03-2076R1. December 16, 2003.
Delventhal, R. A. and N. M. Faget. “Space Shuttle RCS Oxidizer Leak Repair for STS-26[R].” NASA
Johnson Space Center. Document ID 19900009169. May 1, 1989.
Demarchi, E. N. “Hydrazine Leak – Ov-101 Aft Fuselage Wiring.” Rockwell International Internal Memo
392-620-77-619. October 3, 1977.
Dougert, Robert S. “RE: Hypergolic Spill/Fire Report.” E-mail to Robert Dougert. May 18, 2009.
Draus, Thomas. Branch Chief, Shuttle Hypergolics, Hydraulics, & Auxiliary Power Branch. NASA
Kennedy Space Center. Personal interview. July, 21, 2008.
Page 81
NASA/TP-2009-214769
Ezell, Edward C. and Linda N. Ezell. “The Partnership: A History of the Apollo-Soyuz Test Project.”
NASA SP-4209. 1978.
“Fire, Explosion, Compatibility, and Safety Hazards of Hypergols – Hydrazine. American Institute of
Aeronautics and Astronautics Special Project Report. AIAA SP-084-1999. 1999.
“Fire, Explosion, Compatibility, and Safety Hazards of Nitrogen Tetroxide.” American Institute of
Aeronautics and Astronautics Special Project Report. AIAA SP-086-2001. 2001.
Fisher, James and John J. Glisch. “Leak Could Put Discovery on Shelf.” The Orlando Sentinel. July 17,
st
1988, 1 edition: A-1+.
Fisher, James. “Shuttle May Go Back to Hangar for Repair Work.” The Orlando Sentinel. July 19, 1988,
st
1 edition: A-3.
Gehman, Harold W. Jr et al. “Columbia Accident Investigation Board Report Volume I.” August, 2003.
Gehman, Harold W. Jr et al. “Columbia Accident Investigation Board Report Volume II.” October, 2003.
Glisch, John J. “Shuttle Tank Problem May Not Have Had Thorough Check.” The Orlando Sentinel. July
st
20, 1988, 1 edition: A-1+.
Gorham, Richard. “MMH Fire at Test Stand 401.” Honeywell Technology Solutions Inc. Memorandum.
PD00-096. August, 21, 2000.
Hall, George F., Raymond Lake, John H. Storm, and Ross J. Utt. “Fire Protection Research and
Development Requirements Analysis for USAF Space Systems and Ground Support Facilities
Volume I – Fire Protection Operational Requirements Analysis.” Flight Dynamics Directorate
Wright Laboratory Air Force Materiel Command, Tyndall Air Force Base, FL. WL-TR-96-3010.
April, 1995.
Hartsell, Heather. “Top Story – Titan Missile Crisis.” The Arkansas Leader. September 27, 2006.
<http://www.arkansasleader.com/2006/09/top-story-titan-missile-crisis_27.html>.
Heinrich, Manfred M. United Space Alliance Chief Engineer, KSC. Personal Interview. September 17,
2007.
Henderson, J. “Fuel Waste Incident Report.” WSTF Internal Memo. February 21, 1990.
Hill, Gary et. Al. “FCV Vent Port Leakage.” Titan IV SRM Program Technical Interchange Meeting.
Meeting briefing log number 7260. July 31, 1997.
Houston, D. W. “MMH Spillage Investigation.” Rockwell International Laboratory Test Report LTR 3406-
4579. October, 1981.
“Investigation of the WSTF 400-Area Oxidizer Spill Incident.” Johnson Space Center, White Sands New
Mexico, Las Cruces New Mexico. March 29, 1996.
Page 82
NASA/TP-2009-214769
“Investigation Report of the WSTF Peacekeeper Axial Engine Valve Damage.” WSTF-MR-00-0004.
Lyndon B. Johnson Space Center, White Sands Test Facility, Las Cruces New Mexico.
September 15, 2000.
Johnson, B. H. “Failure Analysis of a Fluid Mechanics Check Valve from Orbiter Processing Facility
(OPF) Toxic Vent System.” NASA Engineering Development, Ground Support Office, Materials
Analysis Office, Malfunction Analysis Branch. MAB-221-86. September 30, 1986.
Kamp, G. J. “N2O4 Release at Oxid Farm IPR-6V-1189.” Memorandum for the Record. February 10,
1983.
Kehler, Robert C. “30 SW Operations Review Board Minutes.” Memo for 30 SW/CC, Vandenberg AFB,
CA. August 22, 1997.
Keinath, Richard. “Flash # 008214 – Hypergolic Maintenance Facility (M7-1212) Oxidizer Spill.”
Investigation Committee Slides. March 10, 2006.
Kersey, Judith. ‘Independent Assessment of the OV-103 OMS/RCS Pod Leak and OMS Pod
Processing.” KSC IAI 0988-01. NASA Kennedy Space Center, Florida. February 13, 1989.
Kolcum, Edward H. “NASA Managers Scramble to Limit Leak’s Impact on Discovery Launch.” Aviation
Week & Space Technology. July 25, 1988: 24-25.
Kowalski, Robert R. “Test Cell 844 Manual Valve Failure Mishap Investigation.” NASA Propulsion Test
Office, Johnson Space Center, White Sands Test Facility. WSTF-MR-05-0005. March 14, 2006.
KSC Hazard Report. “Personnel Injury and Flight Hardware Damage Resulting from SRB Hydrazine
Leak or Spill.” Report Number SPC-K13167-86 Rev. B. September 2, 2002.
KSC Hazard Report. “Possible Fuel/Oxidizer Vapor Release During Maintenance/Calibration of Relief
Valves on the MMH/N2O4 Storage Tank.” Report Number SPC-K12186-85 Rev. A. May 22,
2006.
KSC Incident Reporting Information System (IRIS) Database. Incident Numbers: 1985-144-00001, 1986-
021-00001, 1986-210-00001, 1999-232-00007, 2003-224-00006, 2004-162-00002, 2006-027-
00002. 2008.
KSC SSP PRACA Database. “During Attempted Removal of Adapter from Throat of Thruster Liquid
Spilled from Thruster R4D.” PR RP04-9-08-0031 (page 1A). Initiated May 24, 1985.
KSC SSP PRACA Database. “Fuel Waste Barrel Inadvertently Filled With Fuel (MMH). Suspect FV-2
Valve Inadvertently Opened By Lifeline for Traffic in Area.” PR HMFE-0299. Initiated December
12, 1990.
KSC SSP PRACA Database. “FV-1 and FV-2 Valve Handles Extend Into Walk Area and May Be
Inadvertently Cycled on the Fuel FCU.” PR C72-1631-03-001-0015. Initiated March 4, 1991.
KSC SSP PRACA Database. “Liquid MMH Released from Sample Valve at I/F A94209 After Drain and
Evacuation Procedure Completed; S/B No Liquid.” PR FRC4-19-0470. Initiated March 26, 1997.
KSC SSP PRACA Database. “Oxid [N2O4] Cross Country Line Pressure is Decreasing, Indicating an
External Leak.” PR S70-0700-06-002-0019. Initiated November 6, 1997.
KSC SSP PRACA Database. “Small Flame Reported When Sample Gear Was Removed from A78626
Sample Valve N204 Drain When F/H Was Removed Came in Contact With Insulation Which
Burned Momentarily.” PR PADB-0209. Initiated June 24, 1988.
Page 83
NASA/TP-2009-214769
Lance, Renée J. “Approach and Landing Tests Auxiliary Power Unit Subsystem Flight Report.” January,
1978.
McCoy, R. W. “January Interim Report: LT-152 Fire Investigation.” EG & G Florida Interoffice
Correspondence 631-5-008. January 15, 1985.
MER Problem Report (STS-9) STS-9-V-26. November 26, 2007. MER Problem Tracking Database. July
11, 2008. <http://mer.jsc.nasa.gov/merptd/Public/Home.cfm>.
Meyer, R. R. “Incident Report: Aspiration of Oxidizer into the Fuel Aspirator System at Test Stand 401.”
Lockheed Engineering & Sciences Company Interdepartmental Communication. PTD90-119.
March 29, 1990.
Myers, Jeffrey, M.D. “RE: Hyper Spills & Accidents Lessons Learned Report – Toxicology of NO2
Inhalation.” E-mail to Jeffrey Myers and Frank Golan. August 7 2008.
National Aeronautics and Space Administration. “Project: Apollo 7 Press Kit.” Release No: 68-168K.
September 27, 1968.
Nufer, Brian M. “FRC3 HMF Building M7-1212 West Test Cell FRC3 N2O4 Spill on January 9. 2006.”
Incident Summary Report. May, 2006.
Olney, J. N. “OV-101 Hydrazine Exposure Investigation.” Rockwell International Laboratory Test Report
LTR 2291-4466. March, 1978.
Perez, Conrad. “RE: Apollo 7 Spill.” E-mail to Conrad Perez. June 24, 2008.
Porter, Alan R. “RE: 2005 N2H4 Spill Questions.” E-mail to Alan Porter and Kurt Rathgeber. May 29,
2008.
RCS Thruster Heater Power During Ferry Flight Stopovers. Kline, James G. June 2, 1994. NASA
Lessons Learned Website. Lesson Learned Entry 0059. August 1, 2007.
<http://nen.nasa.gov/portal/site/llis>.
Rathgeber, Kurt A., Bruce Havenor, and Steven D. Hornung Ph.D. “Hypergol Systems: Design, Buildup,
and Operation.” NSTC Course 055. January 6, 2006.
Reynaud, Raymond. “Lessons Learned – Quick Disconnect (QD) Oxidizer Exposure Incident.” QARSO-
WSTF Informal Memo. November 20, 1992.
Rodgers, James L., Maj., USAF. ”Titan Oxidizer Leak ORB.” Memorandum for Record for the
th
Department of the Air Force, 45 Space Wing (AFSPC), Patrick AFB, Florida. September 16,
1994.
Sargent, Douglas E. “MMH Spill Mishap Report (January 21, 1986).” Lockheed Space Operations
Company Internal Letter. July 3, 1986.
Schindzielorz, Howard. “Titan IV-A FCV Vent Port Leakage TIM.” Memorandum for Record for the
th
Department of the Air Force, 45 Space Wing (AFSPC), Patrick AFB, Florida. July 31, 1997.
Page 84
NASA/TP-2009-214769
Skaja, Jeffrey T. “RE: Titan IV N2O4 Spill on CCAFS.” E-mail to Jeffrey Skaja. September 4, 2008.
Smitelli, Joseph III. “HMF Oxidizer Exposure.“ Standing Accident Investigation Board Team Report IPB-
04-025-D. June 5, 2004.
Stefanovic, Mike et. Al. “OPF Bay 2 Fuel Spill.” Mini-Investigation Report. February 17, 1997.
Sullivan, J. L. “HMF FRCS #4 Fuel Spill.” Formal Accident Investigation Board Report. KSC Mishap
Report Number USK-97-020. March 26, 1997.
Titan II Missile Explosion. 2008. The Encyclopedia of Arkansas History and Culture. The Central
Arkansas Library System. July 11, 2008. <http://www.encyclopediaofarkansas.net/>.
Torpey, Tom. “Situational Awareness” course notes. NASA Safety Training Center Course Number 034.
January 18, 2006.
Tribe, J. “Incident/Error Review Board Item 70 – Inadvertent Flow of MMH to Atmosphere Resulting in
Ignition/Fire on June 29, 1982.” Rockwell International Internal Letter. L0-JT-82-044. October
29, 1982.
Tribe, J. “Incident 029-83, MMH Spill During FRCS Ferry Plug Removal.” Rockwell International Internal
Letter. L0-JT-82-043. May 2, 1983.
Utsman, T. E. “Monomethylhydrazine (MMH) Spill/Fire, June 29, 1982.” NASA Internal Memo.
VO/Director of Shuttle Operations. Undated.
Walker, J. J. “Evaluation of Materials for N2O4 and Aerozine 50 Compatibility for Chemical Pumps.”
Martin Marietta Corporation Mechanical Engineering Materials Engineering MeME 81/273R.
September, 1981.
Washburn, J. F. “Final Report of the Fire Involving Hypergolic Fuel (MMH) Tanker LT-152.” KSC-E-863.
August, 1984.
Weber, D. A. “Reported Fire During OV-105 L3D Thruster Changeout.” Lockheed Martin Space
Operations Interdepartmental Communication. June, 12, 1995.
What You Should Know about Refrigerants When Purchasing or Repairing a Residential A/C System or
Heat Pump. August 25, 2008. Ozone Layer Depletion – Regulatory Programs. U.S.
Environmental Protection Agency. August 26, 2008.
<http://www.epa.gov/Ozone/title6/phaseout/22phaseout.html>
Wilder, J. “Release of Oxidizer Vapor During Flight Cap Installation on MD338.” Rockwell International
Internal Letter. L0-JW-81-010. July 14, 1981.
Wilder, J. “Incident Report LC-39A.” Rockwell International Internal Letter. L0-JW-81-011. July 15,
1981.
Williams, Wiley E. “LC-39A STS-2 Oxidizer Leak.” Mishap Investigation Committee Final Report.
October 9, 1981.
Page 85
NASA/TP-2009-214769
Page 86
NASA/TP-2009-214769
Page 87
NASA/TP-2009-214769
Page 88
NASA/TP-2009-214769
Page 89
NASA/TP-2009-214769
Page 90
NASA/TP-2009-214769
Page 91
NASA/TP-2009-214769
Page 92
NASA/TP-2009-214769
Page 93
NASA/TP-2009-214769
Page 94
NASA/TP-2009-214769
Page 95
NASA/TP-2009-214769
Page 96
NASA/TP-2009-214769
N2O4 Explosion
Fuel Explosion
N2O4 Injuries
Fuel Injuries
During R&R
N2O4 Liquid
N2O4 Fire
Fuel Fire
MMH
Spilled
A-50
N2H4
Incident Date Quantity Onto/Into
Pad structure
Apollo 7 SPS ~1 to 2
Sept. 1968 and X X R X
N2O4 Spill gal
vehicle
Apollo-Soyuz
Into crew
Astronaut N2O4 7/24/1975 Vapors X X R X X
compartment
Vapor Exposure
OV-101 APU 1 Into vehicle
Cavity Seal N2H4 6/28/1977 ~5 gal aft X X R X X
Spill compartment
N2H4 Spill
OPF1
Following APU Nov. 1979 ~2 gal X X R X
platforms
Hotfire
Page 97
NASA/TP-2009-214769
N2O4 Explosion
Fuel Explosion
N2O4 Injuries
Fuel Injuries
During R&R
N2O4 Liquid
N2O4 Fire
Fuel Fire
MMH
Spilled
A-50
N2H4
Incident Date Quantity Onto/Into
Titan II Explosion
11,140 Bottom of
Following A-50 9/18/1980 X X X X X R R X
gal missile silo
Spill
KSC Incorrect
Flight Cap N2O4 July 1981 Vapors OPF1 X X R R X X
Vapor Release
MMH Exposure
Following
Ground and
Flexhose 7/14/1981 < ½ gal X X R R R X X X
technician
Removal at Pad
Farm
STS-2 OV-102
Into vehicle
Right Pod MMH Fall 1981 ~1 tsp X X X R R X X
pod
Fire
Onto vehicle
STS-2 OV-102 15 to 20
9/22/1981 and pad X X R R X X
N2O4 Spill gal
structure
Page 98
NASA/TP-2009-214769
N2O4 Explosion
Fuel Explosion
N2O4 Injuries
Fuel Injuries
During R&R
N2O4 Liquid
N2O4 Fire
Fuel Fire
MMH
Spilled
A-50
N2H4
Incident Date Quantity Onto/Into
N2O4 Vapor
Release from 2/17/1984 Vapors OPF2 X X R X
Loose Fitting
Page 99
NASA/TP-2009-214769
N2O4 Explosion
Fuel Explosion
N2O4 Injuries
Fuel Injuries
During R&R
N2O4 Liquid
N2O4 Fire
Fuel Fire
MMH
Spilled
A-50
N2H4
Incident Date Quantity Onto/Into
Liquid Trap in
Purge Adapter
5/24/1985 ~1 cup Onto vehicle X X R X X X
Flexhose MMH
Spill
STS-61C OV-102
SRB HPU Onto MLP
12/8/1985 ~3 gal X X R X X
Loading N2H4 surface
Spill
Inadvertent Dry
Ground and
Well Removal 1/21/1986 ~100 gal X X X R R X X
farm sump
MMH Spill
OPF2 Trench
N2H4 Spill and 9/19/1986 < ½ gal OPF2 trench X X X R X X
Fire
Page 100
NASA/TP-2009-214769
N2O4 Explosion
Fuel Explosion
N2O4 Injuries
Fuel Injuries
During R&R
N2O4 Liquid
N2O4 Fire
Fuel Fire
MMH
Spilled
A-50
N2H4
Incident Date Quantity Onto/Into
N2O4 and
Insulation Onto line
6/23/1988 ~2 tbsp X X X R X X
Adhesive Small insulation
Fire
STS-26R OV-103
Into vehicle
N2O4 Tubing Leak 7/14/1988 Vapors X X R R X X
pod
on Vehicle
Aspiration of N2O4
Into fuel vent
into Fuel Vent 3/26/1990 ~2 tbsp X X X R X X X
system
System
HMF Screens Onto drum
~1 to 2
Test Drum MMH 12/7/1990 and HMF X X R R X X
gal
Spill floor
STS-42 OV-103
OPF3 platform
Ferry Plug ~¼ to ¾
2/12/1992 and X X R X X
Removal MMH cup
technician
Spill
Page 101
NASA/TP-2009-214769
N2O4 Explosion
Fuel Explosion
N2O4 Injuries
Fuel Injuries
During R&R
N2O4 Liquid
N2O4 Fire
Fuel Fire
MMH
Spilled
A-50
N2H4
Incident Date Quantity Onto/Into
WSTF Incorrect
Ground and
Flight Cap N2O4 11/4/1992 ~1 cup X X R R R X X
technician
Exposure
STS-69 OV-105
Into vehicle
Left Pod MMH 12/9/1994 < 1 cup X X X R R X X X
pod
Fire
STS-69 OV-105
Into vehicle
Right Pod MMH 5/4/1995 < 1 cup X X X R R X X X
pod
Fire
ORSU Open
Manual Valve 3/1/1996 ~90 gal Ground X X R R X
N2O4 Spill
Page 102
NASA/TP-2009-214769
N2O4 Explosion
Fuel Explosion
N2O4 Injuries
Fuel Injuries
During R&R
N2O4 Liquid
N2O4 Fire
Fuel Fire
MMH
Spilled
A-50
N2H4
Incident Date Quantity Onto/Into
OPF2 platform
OPF2 GSE MMH
2/17/1997 ~1 pint and X X R X X X
Spill
technicians
HMF platform
HMF Sample
3/26/1997 ~¾ cup and X X R X X
Valve MMH Spill
technician
WSTF Pathfinder
~1 to 2
Axial Engine 8/7/2000 Into test cell X X X R X X
gal
Valve Failure
Nearby
WSTF Pathfinder
8/12/2000 ~1 cup hardware and X X X R X X
Small MMH Fire
technician
Page 103
NASA/TP-2009-214769
N2O4 Explosion
Fuel Explosion
N2O4 Injuries
Fuel Injuries
During R&R
N2O4 Liquid
N2O4 Fire
Fuel Fire
MMH
Spilled
A-50
N2H4
Incident Date Quantity Onto/Into
WSTF Pressure
Transducer 3/25/2003 ~3 quarts Test cell table X X X R X X
Explosion
Titan IV N2O4
8/12/2003 ~40 gal Ground X X X X R R X X
Pump Explosion
HMF platform
HMF RP01 N2O4
6/5/2004 ~1.4 gal and X X R R X X
Spill
technician
WSTF N2H4 Spill Into cell and
Following Manual 9/30/2005 ~74 gal containment X X R R X
Valve Failure tank
STS-121 FRC3
1/9/2006 ~2.9 gal HMF platform X X R X X X
N2O4 Spill
Page 104
NASA/TP-2009-214769
Page 105