Apollo Experience Report Guidance and Control Systems Mission Control Programer For Unmanned Apollo Missions As-202, Apollo 4, and Apollo 6
Apollo Experience Report Guidance and Control Systems Mission Control Programer For Unmanned Apollo Missions As-202, Apollo 4, and Apollo 6
Apollo Experience Report Guidance and Control Systems Mission Control Programer For Unmanned Apollo Missions As-202, Apollo 4, and Apollo 6
APOLLO 4, - A N D APOLLO 6
Gene F. Hollozuay
Lyndon B. Johnson Spdce Ceuter
Houston, Texas 77058
5. Supplementary Notes
6. Abmact
Ln unmanned test flight program was required to evaluate the command module heat shield and
h e structural integrity of the command and service module/Saturn launch vehicle. The mission
ontrol programer was developed to provide the unmanned interface between the guidance and navi-
;ation computer and the other spacecraft systems for mission event sequencing and real-time ground
ontrol during missions AS-202, Apollo 4, and Apollo 6. The development of this unmanned pro-
;ramer is traced from the initial concept through the flight test phase. Detailed discussions of hard-
rare development problems a r e given with the resulting solutions. The mission control programer
unctioned correctly without any flight anomalies for all missions. The Apollo 4 mission control
irogramer was reused for the Apollo 6 flight, thus being one of the f i r s t subsystems to be reflown
In an Apollo space flight.
Section Page
SUMMARY ...................................... 1
INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
........................
Backup 0 . 0 5 g Acceleration Sensor 19
T e s t Equipment Certification . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Production Delivery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Qualification Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
...........................
Interface Verification T e s t s 39
Spacecraft Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
iii
Section Page
Time-Delay Failures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
FLIGHT PERFORMANCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
Mission AS-202 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
Apollo 4 Mission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Apollo 6 Mission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Development Schedule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Hardware Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
iv
. ~
TABLES
Table Page
III REAL-TIMECOMMANDS . . . . . . . . . . . . . . . . . . . . . . . . . . 16
FIGURES
Figure Page
V
Figure Page
.......................
(a) Orientation of rework a r e a 39
(b) Closeup of rework area . . ....................... 39
16 Typical diode mounting bracket . . . . . . . . . . . . . . . . . . . . . . . 40
vi
APOLLO EXPER I ENCE REPORT
GUIDANCE AND CONTROL SYSTEMS: MISSION CONTROL PROGRAMER
FOR UNMANNED M I S S IONS AS-202, APOLLO 4, AND APOLLO 6
By Gene F. Holloway
Lyndon B. Johnson Space Center
SUMMARY
The unmanned Apollo missions AS-202, Apollo 4, and Apollo 6 were successful
flights. The flight objectives, which were a prerequisite f o r the manned Apollo
missions, were accomplished f o r each mission. The mission control programer unit
was successfully used f o r all three missions without causing a flight anomaly o r the
l o s s of any functional event f o r which the programer was responsible. The mission
control programer met all the flight and ground test objectives without the loss o r
erroneous indication of any necessary output. The mission control programer did,
however, experience individual component failures during the program. These few
failures were compensated for in the redundant circuit design of the mission control
p r o g r a m e r and did not result in the loss of o r deficiency in any necessary mission
output. Because the mission control programer was designed f o r unmanned missions
and a crew was not present to compensate for possible flight anomalies by switching to
alternate backup systems o r by using alternate mission modes, the mission control
p r o g r a m e r with its sometimes triply redundant paths was required to have higher inher-
ent reliability than other Apollu systems. The Apollo 4 mission control programer was
reflown during the Apollo 6 mission. This unit was the first Apollo system to be
reflown on a space-flight mission.
INTRODUCTION
The structure and heat-shield design of the Apollo command and service module
(CSM) had to be verified under Saturn V launch and lunar- reentry environments before
it could be considered man- rated. The mission control programer (MCP) was devel-
oped by the NASA and the CSM prime contractor to provide the automatic event switch-
ing interface between the input command and control systems (e. g . , the guidance and
navigation (G&N) computer) and the output response systems f o r the Apollo unmanned
test flights. The MCP a l s o provided the real-time ground-control interface for backup
attitude control and sequencing, The objective of this report is to document the MCP
development program f r o m the initial concepts and mission requirements phase; through
the design and manufacturing buildup testing; during the spacecraft installation and
tests; and, finally, through the launch, recovery, and postflight analysis. This Apollo
experience resulted in useful information that should be adapted to the design of future
unmanned space-flight equipment.
The unmanned flight requirements f o r the MCP were identified by the interfacing
subsystem design engineers and ground flight controllers. This report gives a mission
time line for the Apollo 4 mission and demonstrates how this mission was accomplished
using a few key commands from the G&N computer and using the internal logic and
hardwired time delays of the MCP to drive o r switch the interfacing spacecraft systems.
The backup ground-control capability is listed together with a description of each real-
time ground command. An example is given to show how the ground commands could be
used to provide a backup thrust maneuver.
The requirement to test each redundant path o r system in the Apollo launch vehicle
just before launch was a n essential factor in the mission successes of the Apollo Pro-
gram. This report discusses the problems that had to be resolved to perform these
spacecraft redundancy tests on the MCP.
During the MCP development, changes to the spacecraft were approved that
required design changes to the MCP. Some of the spacecraft changes are listed in this
report together with their effect on the MCP design.
A s an aid to the reader, where necessary the original units of measure have been
converted to the equivalent value in the SystGme International d'Unit6s (SI). The SI units
are written first, and the original units are written parenthetically thereafter.
The MCP (fig. 1)consisted of three units: the spacecraft command controller
(SCC; fig. 2), the ground command controller (GCC; similar to the SCC), and the
attitude and deceleration sensor (ADS; fig. 3). These units were located in the space-
craft on a platform assembly mounted in place of the crew couches on the crew couch
shock mounts. The MCP weight was adjusted to approximately 163 kilograms
(360 pounds) so that it could provide the weight necessary to verify the response of the
crew couch s t r u t s to landing impacts. A s shown in figure 4, the keying commands
were supplied to the MCP by the G&N system, the Saturn IVB (S-IVB) instrumentation
unit (IU), the updata link, and the launch control complex. The other interfacing
systems actuated by the MCP output switching functions a r e a l s o shown in figure 4.
2
Main display console
Spacecraft command
_---
1. G&N abort
4. Gimbal motors
5. G&N fail
6. 0.05g
7. Positive-X translation
3
8. Command module (CM) and service module (SM) separation
The first two interface signals, G&N abort and positive- o r negative- Z antenna switch-
ing, were removed from the G&N wiring on spacecraft 017 and 020 because the failures
that could produce either signal were considered t o be single-point failures. The
decision was made that, because the abort signal o r relay closure could be erroneous,
the G&N system computer should not automatically abort a mission. Because real- time
ground commands were available to switch the antennas and because the G&N system
computer controlled the spacecraft attitude during the mission midcourse flightpath,
4
the software task of programing the G&N system computer to switch antennas automat-
ically was considered too costly for the results achieved. The diagram of the connector
interface between the MCP and the G&N system computer (fig. 5) shows that the MCP
provided the 28-V dc power supply for the G&N system relays, and the G&N system
computer provided the logic and relay closures to complete the circuit paths.
-Attitude switch
Attitude switch
Cover
Pushbutton switch
interference
filter
The S-IVB IU provided four keying commands for the MCP. The following list
represents the S-IVB interface keying commands for mission sequencing. Each
command was dually redundant.
1. S-IVB r e s t a r t A
2. S-IVB r e s t a r t B
3. Launch escape tower jettison command A
5
6. Lift-off signal B
Whenever the spacecraft direct- current bus w a s powered, the MCP provided redundant
direct-current power to the S-IVB I U f o r the generation of the discrete sequencing
signals (fig. 6). These keying signals f r o m the IU were used by the MCP logic and
internal time-delay circuits to provide the other required mission sequences.
Reaction control
control system system system indication
A diagram of the logic circuitry of the interface between the MCP and the launch
control and ground support equipment (GSE) is shown in figure 7. This interface pro-
vided launch- control personnel with the capability to d i s a r m the pyrotechnics, switch
off the logic buses, and operate the onboard flight recorders while the spacecraft and
launch vehicle were stacked at the launch site. The program r e s e t signal of this con-
trol interface allowed launch-control personnel to start the MCP; that is, to reset all
latching relays and prepare the MCP logic circuit for lift-off.
6
-
+ ToG&N
c- M
- Flight director attitude indicator aline C-31 D14
J t28 V dc K40
I
t28Vdcarmed
No
connection
c
I
128 V dc
-r- -
% K39
NC z Spare
t 2 8 V dc armed by
CM-SM separate
I- c
C
H
Y
0.059 indication
-". I-
+X translation
C-29
C-27
D14
K38
D12
X K41
NC V K40
+28 V dc armed NC G NC
by launch vehicle/
spacecraft separ ate NC U G&N entry mode C-25 D12
command t2.5 sec
i
T +28 V dc K39
R +28 V dc 1 K38
N 128 V dc K37
-
MCP connector
Figure 5. - Diagram of the G&N system connector interface with the SCC.
7
t To S-mBi n s t r u m e n t unit
S - E B restart A t
I
S -IXB
4
1
4 - J - A
Launch vehicle spacecraft separate start I
S - E B restart B A L
Lift-off signal B
__L-
MCP connector
The Apollo 4 mission events (summarized in table I) are typical of the other
missions and are used as a n example. The planned times f o r certain mission events,
listed in the Apollo 4 Mission Report, varied as much as 28 seconds from the times
that were predicted (table I) less than 3 months before the flight. These changes in the
planned times of certain mission events demonstrate the dynamics of mission planning
and the necessity of designing hardware with the flexibility needed to accomplish these
mission c’hanges.
During the Earth-intersecting coast phase of the Apollo 4 mission, the CSM was
alined in an attitude to achieve a specific thermal gradient (cold soak) a c r o s s the heat
shield. This spacecraft orientation was maintained f o r approximately 4. 5 hours before
the CSM was reoriented f o r the second engine firing of the service propulsion system
(SPS). The sequence of events leading up to and including the second SPS engine ignition
will be discussed as a n example. The detailed sequences to be discussed are between
8
time references tg and tC and between t4 and t6 listed in table I and described in
table 11. These sequences, which were initiated by the G&N system computer, a r e not
listed as specific times (hours, minutes, and seconds) but are given as reference
symbols (tA, tg, t2, etc.). The detailed software programs for the G&N-system
computer established these specific times for the various missions.
%
L- To ground support equipmentllaunch control
+ 2 8 V dc b u s A G Command power A
1
+ U
v
MESC logic bus A safe command
w
Launch control c---ta-- MESC logic buses A&B arm command
input to MCP
4 X Tape recorders record
* L Calibrate telemetry
4 z Program reset
4
+28 V dc
B (2)
Spare launch
No connection
' control pins
c (3) Pyro =pyrotechnic
9
TABLE 1. - APOLLO 4 MISSION DISCRETE EVENTS SUMMARY
~~ ~ ~~
Planned time
Event f r o m lift-off, Actual time,
h r :min: s e c h r : min: s e c
(4
Saturn V ascent to orbit
Guidance reference release 00: 00: 16.70 --
Lift-off 00: 00: 00.00 00: 00: 00.263
Saturn IC (S-IC) inboard engine 00: 02: 15.50 00: 02: 15.52
cut-off
S-IC outboard engine cut-off 00: 02: 32.40 00: 02: 30.77
Saturn I1 (S-II) engine ignition 00: 02: 35.20 --
S-IC interstage jettison 00: 03: 04.35 --
Launch escape system jettison 00: 03: 08.35 --
b
S-I1 engine cut-off 00: 08: 39.55 00: 08: 39.76
b
S-IVB engine ignition 00: 08: 44.05 00: 08: 40.72
b
S-IVB engine cut-off 00: 11:05.40 00: 11: 05.64
Earth parking orbit
Start Earth parking orbit 00: 11: 15.60 00: 11: 15.6
Start second orbit revolution 01: 38: 20.00 --
Second S-IVB firing
b
S-IVB engine ignition 03: 11: 54.50 03: 11: 26.6
b 03: 16:26.3
S-IVB engine cut-off 03: 17: 12.53
Coast to f i r s t service propulsion system
(SPS) firing
Begin reorientation to cold-soak 03: 17: 27.71
attitude
End of reorientation to cold-soak 03: 20: 42.81
atti tude
CSM/S-IVB separation 03: 27: 14.43 03: 26: 28.2
Planned time
Event f r o m lift-off, Actual time,
h r : min: sec hr :min: sec
(4
Begin reorientation to f i r s t SPS 03: 27: 22.73
ignition attitude
End of reorientation to f i r s t SPS 03: 27: 51.81
ignition attitude
F i r s t SPS firing
b
SPS engine ignition 03: 28: 52.73 03: 28: 06.6
b
SPS engine cut-off 03: 29: 18.93 03: 28: 22.6
Earth intersecting coast
Begin reorientation to cold-soak 03: 29: 24.68
attitude
End of reorientation to cold-soak 03:29: 53.76
attitude
Apogee 05:49:04.32 --
Begin reorientation to second SPS 08: 01: 36.75 = tB (note c )
ignition attitude
End of reorientation to second SPS 08: 02: 01.05 = tC (note c )
ignition attitude
Reaction control system (RCS) 08: 14: 40.42
t h r u s t e r s on
Second SPS firing
b
Second SPS engine ignition 08: 15: 10.42 08: 10: 54.8
= t4 (note c )
b
Second SPS ,engine cut-off 08: 19: 34.40 08: 15: 35.4
= t6 (note c)
11
TABLE 1.- APOLLO 4 MISSION DISCRETE EVENTS SUMMARY - Concluded
Planned time
from lift-off, Actual time,
Event h r : min: sec h r :min: s e c
(a)
~~~ ~
Preentry sequence
Begin reorientation to CM/SM 08: 20: 12.97
separation attitude
End of attitude orientation, coast 08: 20: 54.01
to CM/SM separation
CM/SM separation 08: 22: 07.85 08: 18: 02.6
Start CM attitude orientation for 08: 22: 12.85 --
entry
End of attitude orientation, coast 08: 22: 36.03 08: 18: 06.28
to entry
Atmospheric entry
0.05g indication -- 08: 19: 56.28
121 9 2 0 - m (400 000 f t ) altitude 08: 23: 35.02 --
Enter S-band blackout 08: 23: 57.00 --
Enter C-band blackout 08: 24: 01.00
Exit C-band blackout 08: 25: 55.00
Exit S-band blackout 08: 26: 19.00
Enter S-band blackout 08: 30: 15.00
Exit S-band blackout 08: 31: 47.00 --
Drogue-parachute deploymen. 08: 35: 39.00 08: 31: 18.6
Main-parachute deployment 08: 36: 27.00 --
CM landing 08: 41: 25.00 --
13
TABLE II. - NOMINAL MISSION SEQUENCE OF EVENTS
Time
reference 1
I
Initiated
by- 1
I
MCP
function 1
I
Function
I
I
output
to-
14
The MCP was designed with the specification that the initiation times for partic-
ular keying and sequencing commands f o r performing various mission functions could
be changed from mission to mission. However, the detailed integrated sequence of
events to accomplish any particular mission function would remain consistent f o r all
missions. Table I1 lists the functions required to reorient the spacecraft to the second
SPS engine ignition attitude and the functions required to initiate and complete the
second thrust maneuver. The M C P time delays a r e shown in the time-reference
column. For example, "t + 1.0 sec" indicates that the "yaw 1 gimbal motor start"
3
signal from the MCP to the SPS gimbal actuator motor occurred 1 . 0 second after the
G&N system computer had given the "gimbal motors ON" signal (t3) to the MCP. The
time-delay units were hardwired, potted plug- in modules that were hermetically sealed
in a metal case. Several time-delay selections were available f o r a given module base
connection size. Because of the high-start- current requirements of the motors, the
gimbal motors were turned on at 0. 5-second intervals to prevent an electrical overload.
Table I1 gives a function- by-function description of the spacecraft system activity
required to perform an SPS thrust maneuver. A similar functional listing can be
obtained f o r all the required mission events, including abort sequencing.
At time tg, the MCP turned on the flight qualification recorder and the data-
storage- equipment recorder. The sequences of events that were considered the most
critical or of the highest priority and that were to be recorded varied significantly
f r o m mission to mission. The recorders and cameras had a limited tape and film
capacity; and timed on- off sequences, which varied significantly from mission to
mission, were necessary to obtain only the most important data. These changes in
sequence times required MCP hardware changes. Usually, the times varied s o much
that different connector interface circuits had to be selected f o r the keying commands
(e. g., a command for a second SPS firing gimbal motors off instead of a command for
S-IVB/spacecraft separation). These MCP hardware modifications were costly in
t e r m s of money and schedule time, requiring new engineering drawings, specification
revisions, t e s t equipment modifications, recertification of the test equipment, and
reacceptance testing of the flight hardware. In future manned o r unmanned develop-
mental flight programs, strong emphasis should be given to the developmental instru-
mentation interface with the spacecraft systems. For launch vehicles or spacecraft
that contain flight computers, the instrumentation used to monitor flight events during
the developmental program should be designed so that the changes of instrumentation
sequences from mission to mission can be placed in the erasable portion of the comput-
er memory. If this procedure were followed, the sequences could be quickly and cheap-
ly modified in real time.
15
TABLE 111. - REAL-TIME COMMANDS
Real- ti me
command Title
number
16
TABLE 111. - REAL-TIME COMMANDS - Continued
Real- ti me
command Title
number
34 Propellant O N SM quad C
35 Propellant ON SM quad D
40 Launch escape tower jettison
41 G&N f a i l
42 G&N f a i l inhibit
43 Reset real-time command numbers 41 to 42
44 Roll rate backup
45 Pitch r a t e backup
46 Yaw rate backup
47 Flight director attitude indicator aline
50 Reset real-time commands 44 to 47
51 Negative- Z antenna ON (very- high-frequency (vhf)
scimitar only)
52 Positive- Z antenna ON (vhf scimitar only)
54 Roll A and C channel disable
55 Roll B and D channel disable
56 Pitch channel disable
57 Yaw channel disable
60 Reset real-time command numbers 54 to 57
61 CM and SM separation
62 Updata link S-band receiver select
63 Updat a link ultrahigh- f r equency receiver
select
64 Hydrogen tank 2 heater fan ON
65 Oxygen tank 2 heater fan ON
66 Hydrogen tank 1 heater fan ON
67 Oxygen tank 1 heater fan ON
70 Reset real-time command numbers 64 to 67
17
TABLE 111. - REAL-TIME COMMANDS - Concluded
~~
Real-time
command Title
number
3. Send real-time command 11, "direct thrust ON, I t at the desired firing time to
automatically s t a r t the gimbal motors in sequence and to initiate the firing.
4. Monitor the spacecraft trajectory by using the Mission Control Center real-
time tracking data.
18
7. Send real-time command 61, "CM and SM separation, I ' to a r m the m a s t e r
events sequence- controller (MESC) logic circuitry, to a r m the pyrotechnic devices,
and to initiate the separation sequence.
Number Title
19
-
TABLE J3J. NOMINAL MISSION SEQUENCE OF EVENTS FROM ENTRY TO LANDING
i1
to (about 28 min G &N X 0.05g ON = 88 400 m (z290 000 ft) MCP
before landing)
0.05g signal (backed up by the MCP sc s
0.05g backup function)
Earth landing system (ELS) activate A ELS
ELS activate B ELS
MESC 7620-m (25 000 ft) barometric switch armed ELS
Switch to negative- 2 antenna T/C
7620-m (25 000 ft) altitude
20
Sequencing Postlanding Recovery Aids
Another function of the MCP w a s to sequence the postlanding recovery aids for the
unmanned missions. The correct performance of these functions was necessary to
ensure the recovery of the spacecraft after landing was successfully achieved. The
sequence of events after landing (table V) w a s initiated by the impact of the spacecraft
on the water. The impact was sensed by triply redundant switch accelerometers in the
ADS unit of the MCP. The MCP was also required to test and certify the uprighting
system of the spacecraft before a manned flight. The ADS unit contained triply redun-
dant attitude indicators that could sense whether the spacecraft w a s floating apex up
(stable I) o r apex down (stable 11). If a stable I1 signal had been indicated by the attitude
switches (table V), logic circuits i n the SCC would have relayed a signal to the upright-
ing system to inflate the flotation bags.
~-
Time
reference
Function
I Output
to-
Nominal events
~~
Impact landing
to
Main-parachute disconnect A ELS
Main-parachute disconnect B ELS
Arm attitude indicator MCP
Connect entry batteries to F&PL bus EPS
Connect auxiliary batteries 1 and 3 EPS
to F&PL bus
Connect auxiliary batteries 2 and 3 EPS
to F&PL bus
t
0
+ 11 s e c Remove entry batteries from main buses EPS
Deploy high-frequency (hf) recovery T/C
antenna (stable I only)
MESC logic bus A safe MESC
MESC logic bus B safe MESC
Flashing light ON T/C
hf transceiver ON (stable I only) T/C
Circuit breaker 45 OPEN
21
TABLE V. - NOMINAL MISSION RECOVERY SEQUENCE O F EVENTS - Concluded
[All events a r e MCP functions initiated by the MCP.]
Time
reference Function
1 output
to-
tl
Flotation pumps O F F URS”
hf transceiver ON T/C
Deploy hf recovery antenna T/C
tl + 1.0 s e c MESC pyrotechnic bus A safe MESC
MESC pyrotechnic bus B safe MESC
~ ~
hf transceiver O F F T/C
to + 60 sec Flotation pumps ON URS
Flotation bag 1 f i l l URS
to + 360 s e c Flotation bag 1 O F F URS
Flotation bag 2 f i l l URS
to + 660 sec Flotation bag 2 O F F URS
Flotation bag 3 f i l l URS
to + 960 sec Flotation bag 3 OFF URS
Flotation pump O F F URS
%Jprighting system.
22
DESIGN USING EXISTING TECHNOLOGY
The critical development schedules for the MCP required that existing electronic
technology be used. Components that had been previously qualified on other missile o r
space programs were selected whenever practicable.
I
and switching circuitry. The relays
operated at 28 V dc and had a 2-, 3-, or
10-ampere current rating.
outputT.. 28Vdc
The redundancy requirements of the
MCP were classified into four categories. I Relay A
Input
1. Simplex (not redundant) - The 1 3
MCP output o r real- time- command func- 5 1
tion may fail either ON o r OFF because of
a single MCP component failure ( c ) Dual parallel (redundant).
(fig. 8(a)).
23
The redundancy requirements for the MCP design were established by using these
four categories. A request for a definition of the requirements of the system interface
was submitted to the appropriate engineering design groups, and specific redundancy
requirements were obtained on an event- by-event basis f o r numerous potential mis-
sions. The design of the MCP was then established consistent with these mission
redundancy requirements.
The redundancy options that were used in the MCP are shown in figure 8. Exam-
ples of equivalent redundancy could also be illustrated within the MCP,showing the use
of time delays, capacitors, diodes, et cetera. The relays are used f o r illustration
because they represent the majority of the components in the MCP.
The circuits in figure 8 are shown with relay contacts configured in the normally
open state. Similar redundant configurations are used with the relay contacts in a
normally closed state. Both momentary and latching relays were used in the MCP
design. Momentary relays remain switched into the changed- state configuration only
as long as the switching signal is applied to the solenoid. Latching relays remain in
the switched configuration until a n additional reset switching signal is applied to the
r e s e t solenoid of the relay.
A triply redundant grounding network was used throughout the cable- harness and
panel-harness assemblies (fig, 9) to provide electrical grounds for the MCP. This
grounding scheme was important in accomplishing the bench tests and spacecraft t e s t s
that verified the redundant components within the MCP. During tests, these grounds
(Gl, G2, and G ) were alternately cycled (opened and closed) o r cycled in combinations
3
(G G , G2G3, o r G1G2) to isolate and verify the operation of specific redundant paths.
1 3
For example, in figure 8(d), assume relay A operates with ground G * relay B with
1’
ground G ; and relay C with ground G3. The redundant paths would be verified as
2
follows.
3. Step 2 verifies the center path shown in figure 8(d). The top and bottom cir-
cuits remain open because relay C has no ground to complete its circuit and does not
close.
6. Step 5 verifies the bottom path shown on figure 8(d). The two top paths remain
open because relay A has no electrical ground to activate its solenoid circuits and the
contacts do not close.
7. Relay A in the middle path does not have its contacts failed in a closed position
in step 3, because in step 6 the middle path opened. If, in step 6, the middle row of con-
tacts had not opened, the failure of relay A in a closed position would have been indicated.
24
Figure 9.- Cable assembly wire harness.
This method can be continued until each redundant function is verified. This
redundancy was an important p a r t of the Apollo Program, because the proper function-
ing of each redundant circuit path had to be verified just before the spacecraft was
launched. The procedure of checking redundant circuitry before launch was used for
the Mercury spacecraft and was continued for the Apollo spacecraft. In general, this
verification of redundant circuit paths was a simple task for the Apollo spacecraft,
because most systems were designed to be dually redundant (system A and system B).
The power could be removed from either system to verify the proper functioning of the
companion system. However, checking the redundant circuitry in the MCP became a
difficult and tedious job because of the many complex series-parallel circuit paths.
As shown in figure 2, the SCC used 19 printed wiring boards (control assemblies).
The detailed logic circuits, relays, time-delay circuits, and other components were
plugged into these control assemblies; the components of this circuitry were standard-
ized and interchangeable. For example, a 3-ampere latching relay could be inter-
changed with a 3-ampere momentary relay, or a 15-second time-delay device could be
interchanged with a 60-second time-delay device (figs. 10 and 11). Great c a r e had to
be taken by the manufacturing personnel when removing a componept that had previously
been mounted and soldered to the printed wiring board; otherwise, the metallic track
could be lifted from the board or damaged. The control assemblies used f o r the vari-
ous MCP systems were also standardized and interchangeable. For example, control
assembly 6 in MCP system 2 and the similar control assembly inMCP system 4 could
be interchanged to resolve a problem with solder closeout relays.
The SCC had 15 connectors to meet the various interface requirements and to
provide sufficient test points f o r ground tests. The unit had a ground-shorting connec-
t o r and three GSE connectors that were instrumented for the box-level bench tests.
The internal grounds could be automatically applied and removed while the operation of
various components was being verified on the bench test console.
25
Figure 10. - Printed wiring board with components.
26
Figure 11. - Bracket showing relays and time-display mountings.
3. The gimbal motor on-off times were changed from mission to mission to pre-
vent the actuator clutches from overheating.
4. A time delay was added in the MCP to prevent damage to the high-frequency
antenna by not allowing the antenna to deploy before the spacecraft was in an apex-up
attitude in the water.
5. Before the mission, the SPS engine gimbal positions were predicted for each
firing during the mission. These positions were preset in the MCP to prevent large
gimbal position changes and large transients during the firing initialization. Each space-
craft had different center- of- gravity requirements at the various firing times; thus,
each spacecraft required different gimbal position settings that necessitated modification
of the MCP system.
27
New interface
6. On-off sequences f o r the tape re-
, c o r d e r s and c a m e r a s were changed f o r each
C6AlJ13
-L
: I
I
-
mission.
Ground command G-1
control le r dc negati ve I
E a
Ground command
controller dc negative
G-2 0 These few examples of the hardware changes
I
I Jmade to the MCP indicate the flexibility
I
I
Ground command G-3
controller dc negative
required of a developmental flight system
such as the MCP. For example, before the
design of the MCP was complete, preplan-
ning should have determined that the gimbal
position settings would vary from mission to
controller dc negative mission and that the hardware should be
designed s o that a technician could change
controller dc negative
the settings without opening the SCC. When-
Spacecraft command c-3 z e v e r this unit was opened, a complete reac-
controller dc negative
-A ceptance test was required. In future
%+J
The GCC used component and wiring redundancy s i m i l a r to that previously de-
scribedfor the SCC. The series-redundant circuitry (fig. 8) was the most commonly
used circuit logic; however, parallel- redundant circuitry was used f o r processing the
reset real-time command, and series-parallel- redundant circuitry was used f o r proc-
essing the abort command.
28
The r e s e t real-time commands 05, 13, 23, 43, 50, 60, and 70 were necessary
because, once the GCC relay driver received a minimum-value current pulse from the
updata link, the associated latching relays were activated. The real- time command
could be removed o r canceled only by sending a reset command. Some real-time com-
mands used momentary relays (e. g., positive- Z antenna ON). These momentary
relays were on as long as the command was being transmitted and off at all other times.
The use of latching relays saved the electrical power that would have been required to
hold the relay solenoid in the activated state and amounted to considerable power savings
for events that would be on for long periods.
4. A pivot shaft and pivot f r a m e for ground test of the attitude indicators
5. Push- to-test switches for ground testing the 0.05g and landing accelerometers
1. The 0.05g accelerometers were armed by a signal from the SCC at the time of
CM and SM separation. These accelerometers were designed to trigger at decelerations
of 0. l g to 0. 5g, a higher deceleration value than the 0.05g value furnished by the G&N
system. In June 1966, the 0.4g spread in the tolerance of the backup deceleration sen-
s o r was recognized as possibly causing a wide deviation between the actual and planned
spacecraft landing points when the backup signal w a s used. A specific test was then
added to the box-level acceptance test to measure and record the exact deceleration
level of this sensor. For spacecraft 017, this deceleration value was 0.29g * 0.04g f o r
initiating the backup 0.05g signal. The 0.05g signal would be automatically overridden
by the ADS in case of a skipout trajectory. The *O. 04g tolerance could not be discarded
because the accelerometers were temperature sensitive and the precise flight tempera-
t u r e s were not defined. The trigger point was better defined, however, at an order of
magnitude closer than the initial values.
2. The three impact switches were armed by a signal relayed from the SCC when
it sensed a n altitude of 3658 meters (12 000 feet). The impact switches were designed
not to trigger f o r impact pulses l e s s than 4.7g and to trigger f o r values of approximately
5g and above. A push-to-test switch was provided f o r each impact switch f o r ground-
test purposes.
29
3. The three attitude sensors were a r m e d by the impact deceleration pulse,
These sensors indicated stable I whenever the apex of the spacecraft was approximately
k65" from a n upright position. When the apex of the spacecraft dropped below the
65" point, stable I1 was indicated by the sensors. Additional stable I1 functions of the
sensors a r e given in table V. These attitude sensors could be tested in the spacecraft
by loosening a hexagonal nut (fig. 3) and pivoting the sensors to effect a change in atti-
tude signal.
The manufacturing test sets were essential in the test activity associated with the pro-
duction and assembly of the MCP control assemblies and printed wire networks. These
test sets performed satisfactorily and supported the program in a timely manner.
The manual bench test console verified the operational status of each redundant
component in the M C P during acceptance tests and other box-level tests. This bench
console required that the input signal be switched manually at the times required by the
test specifications. Groups of test points (e. g . , 60 test points) were collectively mon-
itored and, if no anomaly occurred, that test zone of redundant elements within the M C P
was considered satisfactory. This MTE was s i m i l a r to the equipment developed f o r the
control programer in spacecraft 009 (AS- 201 mission) and was completed on Novem-
b e r 14, 1965, in time to support the initial breadboard and e a r l y MCP prototype
deliveries.
The MTE required approximately three times as long to complete a test run as
did the automatic FTE; however, the MTE was sufficiently simple that the equipment
could be certified and debugged in a timely manner and could be reconfigured f o r com-
patibility with changes in the flight hardware.
The automatic FTE was used for the postenvironmental functional t e s t s and the
MCP systems tests. This test equipment included the following.
1. A punched-tape reader to provide the input stimuli with the associated power
supplies and signal- conditioning equipment
2. A s e r i e s of internal logic circuits to control the switching and route the sig-
nals to the correct M C P area
30
3. A master clock to control the timing of the input signals and to provide a time
comparison of the MCP response
The automatic FTE was primarily used to support qualification testing. A series
of test tapes was prepared to support the environmental and postenvironmental func-
tional tests as follows.
1. Environmental functional t e s t s
The requirement existed to automate the test and sequencing of the MCP while it
was operating in the qualification- test environment. F o r example, during the vacuum
test, an abbreviated-time simulated mission was performed while the MCP was in the
4-hour soak period of the vacuum environment. This requirement would have been
impossible to achieve with the MTE because 48 hours would be required to sequence the
MCP through all the programer functions manually. As a result, the requirement f o r a
4-hour vacuum soak would have to be exceeded. However, if the 4-hour vacuum soak
were retained, the number of functions that could be manually sequenced would be so
limited that only a small p a r t of the MCP internal logic circuits could be tested.
The development and certification of the elaborate and complex FTE within the
allotted schedule period caused considerable difficulty. A 4-month delay in the start
of qualification of the GCC and SCC was generally attributed to problems in certifying
the test equipment, the test specifications, and the test tapes. The schedule problem
concerning certification of the FTE was related to the original design concept and the
planned method of test. Considering the critical development schedules and the small
number of unmanned systems to be delivered, the test equipment concept was much too
complex and automation was overly emphasized.
1. The test equipment did not verify functional paths within the MCP but checked
out zones or groups of components; therefore, when a hardware change was incorporated
in the MCP, a compatible change was difficult to incorporate in the related component
group of the test equipment. Hardware changes also caused difficulty in updating the
test specifications.
31
2. The tape reader had no reliable method of performing an internal verification
or self-check. If a part of the tape message was missed, it was difficult to determine
whether the problem was in the reader, the test equipment, the MCP, o r some other
system or component.
3. A reliable method was not developed to revise only specific sections of the
test tapes to reflect hardware modifications. A reprograming effort involving the entire
test sequence seemed to be required. The test tape could not be cut and spliced; there-
fore, a new tape had to be generated to include the updated test section. A s a result,
the manpower requirements for test equipment programing were increased whenever
the flight hardware was changed.
Because of these test equipment reprograming delays, the FTE was not used to
any great extent in supporting the M C P reacceptance tests following design modifica-
tions. The MCP redundancy test performed in the spacecraft provided a sufficient
confidence level, and a systems-level functional acceptance test at the vendor was not
required. The FTE was not reprogramed and reconfigured to reflect the numerous
MCP hardware changes. The engineering time was more efficiently used in actually
performing the vendor box-level acceptance t e s t s on the slower MTE than in preparing
the automatic FTE to perform the MCP system-level test.
On June 25, 1964, the Apollo prime contractor was notified to develop a programer
with the capability to conduct the unmanned missions AS- 201, AS- 202, AS- 501, and
AS-502. The original schedule f o r the MCP installation into spacecraft 011 at the con-
tractor's facility was January 13, 1966. The MCP development team had 19 months to
design, build, test, and deliver the first flight system. The following paragraphs
describe the most significant milestones concerning this development.
32
Breadboard and Prototype Development
The following schedule was achieved and indicates the compressed and critical
nature of delivery milestones f o r the MCP.
Although the design configuration freeze was dated October 28, the following sig-
nificant changes to the MCP design were approved on November 8; therefore, the
configuration was not really frozen.
1. The on and off times of the flight-qualification tape recorders were changed
and required wiring changes in the MCP.
A maximum of 1 month was scheduled between delivery of the breadboard and the
first prototype. The t e r m "breadboard" cannot be used in the sense that the breadboard
was a device to be tested and evaluated, with the results of the evaluations being fed
back as design improvements. The rigorous acceptance tests and inspection-approval
criteria that normally constrain development did not apply to this breadboard unit;
therefore, the manufacturer could produce the unit as a working device to help in the
test equipment development and certification. The prototype unit used the same produc-
tion manufacturing and assembly techniques as the flight units. The f i r s t prototype was
delivered to the spacecraft contractor for simulation testing and interface verification
testing. The combined systems t e s t s and simulations, using the first prototype unit,
uncovered the problem of the MCP relay drivers triggering on noise. These evaluation
t e s t s were also valuable in establishing a redundancy checkout scheme f o r the MCP
while it was installed in the spacecraft.
33
voltages. At the beginning of the Apollo Program, one MCP unit was scheduled f o r
environmental qualification tests and another unit for e m i qualification tests. The e m i
test and success values were difficult to establish on a black-box level because the
interference is a n interrelated- systems problem. Therefore, the requirement f o r e m i
qualification t e s t s on the black-box level was eliminated before the MCP qualification
tests were scheduled. An overall emi test scheme was t o be established on the
spacecraft- test level.
Item Date
Certification of the FTE was important in that this certification was a constraint
to the start of the systems test portion of the MCP qualification program. The F T E
certification, o r development testing, could not begin without an MCP test article t o
process the responses to the test input signals. The MCP breadboard system, delivered
in November 1965, was used in this development. This late delivery allowed only
6 months f o r development and certification testing of the FTE. This length of time
was extremely short f o r testing, debugging, and certifying a test equipment system of
this complexity. However, the original schedules allowed only a 2-month period from
breadboard delivery to certification completion and qualification test start. This
period was not sufficient to achieve the test equipment certification; therefore, the quali-
fication s t a r t date was extended by 4 months.
The certification of the test equipment was achieved by using a production proto-
type MCP unit that was essentially identical to the qualification unit to be tested later.
First, an acceptance test using manual test methods was performed on the prototype,
and each redundant function was verified to be operating. This unit was then used as a
test equipment certification unit, and the s a m e test specifications were used. If every
test function was processed through the certification unit and was recorded by the FTE
with no anomalies, the test function was certified. If a n anomaly occurred, then a n
analysis had to be performed t o determine whether the test equipment or the certifica-
tion unit had malfunctioned. This step-by- step method was demanding and time con-
suming, but the FTE was finally certified.
34
Production Delivery
The following schedule was achieved f o r the M C P production unit deliveries and
modifications.
Of the six production units delivered, units 1, 3, and 4 required several design
modifications to make them compatible with the M C P design configuration f o r space-
craft 017 and 020. The design changes were incorporated in production units 5 and 6
before delivery. The qualification unit 2 did not require modification because the design
modifications did not require requalification testing. The MCP supported the spacecraft
delivery and test schedule dates; however, some of the design changes and rework had
to be accomplished during the idle vehicle test periods. For example, the rework was
started after the MCP finished supporting the integrated systems test at the spacecraft
contractor's facility and was completed before the next requirement to support tests in
the vehicle at the NASA John F. Kennedy Space Center (KSC). Ideally, the spacecraft
connectors would not have been disturbed, and the MCP would have been delivered to
the KSC while installed in the spacecraft.
Qualification Tests
The following key schedule dates describe the qualification test program.
Item Date
35
Item Date
Qualification test completion April 23, 1966
Qualification report release May 31, 1966
1. Vacuum 8. Acceleration
2. Oxidation 9. Factory test equipment test
3. Humidity 10. Manufacturing rework
4. Resistance 11. Manual test equipment test and specification certification
5. Random vibration 12. Factory test equipment test and specification certification
6. High temperature 13. Life test vibration
Qualification testing, 7. Shock 14. Real-time mission simulated mission
system 1 15. Manual test equipment tests functional
Ground command controller
and spacecraft command
controller
Qualification testing,
system 2
Spacecrafl command
contro Iler
Ground command
controller [4111213[ 11 I 15 I 12 IlOld 12 I 13 I 12 ] m
Attitude and
deceleration sensor
L11
m
I 1 I I I I 1 1 I I I I I 1
17 20 25 '1 5 10 15 20 25 I 1 5 10 15 20 25
February March Apri I
The ADS package of MCP production unit 2 was subjected to the qualification test
environments. Because of the simple design of this sensor package, all the environ-
mental tests, except shock, were completed in 1 week. Because this package also had
36
t o support the life test, the shock environmental test was postponed until the end of
qualification testing. The sensor package successfully completed the qualification tests
with no anomalies, no visible physical damage, and no operational degradation,
The MCP production unit 2 (qualification test unit 2) was initially ready to begin
qualification testing on February 26, 1966. However, the FTE either had not been
completed or was not certified. Because of the critical schedule requirements, the
qualification tests were started, using only a few manually initiated commands f o r each
postenvironmental verification of the MCP. No failures were detected during the initial
vacuum, oxidation, o r humidity testing.
After the humidity test, the qualification test packages were to be given a complete
package functional test, using the MTE. This equipment had been used earlier to com-
plete the testing of qualification unit 2, but the new test specifications required box-level
testing of redundant circuitry. These specifications had not been checked out against
any package o r with the MTE. When the postenvironmental (vacuum, oxidation, and
humidity) testing was attempted, numerous problems were encountered and too many
unknowns (such as MTE, test specifications, and MCP) were involved. As a result, the
period between March 2 and April 2, 1966, was used to debug and certify the specifica-
tions and the MTE, to retest the MCP, and to check out the functional test tapes f o r the
FTE. During this period, failures were detected in the packages; some failures were
due to manufacturing e r r o r s not previously tested in the redundant circuits during sell-
off; others were induced by the MTE. These failures reemphasized the critical require-
ment of entering a qualification test program with a good baseline; that is, with certified
test equipment, verified procedures, and adequate specifications.
The MCP production unit 3 (qualification test unit 1) entered the vibration environ-
ment portion of the qualification test on April 2, 1966. The vibration effects to be
investigated were the resonances of each package (resonance search) and the suscepti-
bility of the MCP to random vibration. The postvibration physical inspection of the MCP
indicated 35 instances of fractured solder joints on the pins of the SCC control assembly
connector boards (fig. 14). The GCC had 10 loose o r broken solder joints around
s i m i l a r pins. However, no functional test failures were attributed to the solder frac-
t u r e s around the pins. The corrective action f o r the cracked-solder- joint problem
included soldering the terminals on both sides of the circuit strip (fig. 15) and adding
a bracket to improve the wire- bundle routing. The corrective action was successful,
and the problem did not r e c u r during future vibration tests.
37
(a) Example 1.
(c) Example 3.
(b) Example 2.
38
(a) Orientation of rework area. (b) Closeup of rework area.
The second significant problem was detected during the humidity and posthumidity
functional test portions of the qualification program. The reverse impedance of the
MCP diode quads was below the specification limits after the 16-hour humidity test.
These impedances were within the specified value (greater than 700 kilohms) after about
1 hour of drying. The diodes in the SCC were affected after being exposed to
95 + 5 percent relative humidity; but, for normal unmanned flight, humidity was not
expected to be a problem. The corrective action was to provide added protection by
applying polyurethane (polycoat) to the control assemblies containing the diode quads
(fig. 16). On April 22, 1966, after the polyurethane was applied to the control assem-
blies in qualification unit 2, the unit was retested in the humidity environment and
satisfactorily met the specifications. The qualification tests were completed on
April 23, 1966, and the test report was released on May 31, 1966.
39
5. Identification of the system interface incompatibilities
The MCP prototype unit 1 was delivered to the spacecraft contractor on Decem-
b e r 3, 1965. This unit was first checked to verify electrical interface and compatibility
with the spacecraft electrical power system. Unit 1 was then subjected to several
different interface tests with individual spacecraft systems, such as the communications
system and the MESC. Finally, the MCP prototype was tested, along with several
other systems, in the guidance and control laboratory during the combined systems
dynamic verification tests. The combined systems-test setup and the use of toggle
switches to switch the internal grounds (G1, G2, and G3) of the MCP suggested a method
for testing redundancy of the MCP while in the spacecraft.
40
Spacecraft Tests
The M C P was installed in the spacecraft before the combined and integrated sys-
t e m s tests were started at the spacecraft contractor's facility. The MCP served as the
interfacing unit between the G&N system and other input stimulus sources and the space-
craft flight systems that actually activated and performed the desired spacecraft output
functions during the ground tests and also during flight. The MCP had no flight or
spacecraft test measurement allocation, even though it had 10 connectors with over
500 measurement pins readily available f o r bench t e s t s at the factory. The rationale
f o r having no MCP flight measurements nor ground test measurements was that the un-
manned vehicles would be instrumented and tested the same as the manned vehicles.
Therefore, the functional operation of the M C P was determined by observing the func-
tional operation of the related output systems that were instrumented. This rationale
would have been adequate if the MCP processed programer signals through single func-
tional paths. However, the MCP contained numerous s e r i e s - redundant and parallel-
redundant paths (as previously described).
The Apollo Program had a requirement that each redundant path be verified as
functioning properly just before launch. This requirement was interpreted to mean that
the MCP, even though it was f o r unmanned flights, had to have its redundant paths veri-
fied in the spacecraft just before launch. The following schedule indicates the time
required to implement the MCP redundancy tests.
Event Date
41
During the l - y e a r period between the original directive and the final processing of
procedures for performing this test, the contractor maintained that the MCP redundancy
tests would not be advantageous f o r the following reasons.
1. A 50-man-month effort in programing cost f o r acceptance checkout equipment
could be saved.
2. A saving of 120 hours of spacecraft test time would result, compared with the
12 hours required f o r bench test equipment.
For the spacecraft 011 MCP redundancy test a t the KSC, a special breakout box
was designed to interface with the MCP ground-shorting connector. By using jumper
wires on the breakout box, the test team could cycle the internal grounds (G1, G2, and
G3) of the MCP. Figure 1 2 shows the MCP/acceptance checkout equipment interface
f o r the spacecraft 017 and 020 redundancy test; acceptance-checkout-equipment relays
were furnished and automatically sequenced to cycle the grounds. The jumper wires
that were opened and closed manually for spacecraft 011 served the s a m e purpose as the
acceptance- checkout- equipment relays but required more test time. The test actually
consisted of three abbreviated mission- time-line test sequences with the required space-
craft systems powered up; the normal prelaunch countdown was performed but was
stopped just before lift-off. One run through the abbreviated mission sequence of events
was made with the appropriate opening and closing of internal ground G1. Two s i m i l a r
runs were then made with internal grounds G2 and G3 being cycled open and closed.
These runs were to verify that the redundant paths of the MCP were operative while the
unit was in the spacecraft. These redundancy tests were performed at the KSC for
spacecraft 011, 017, and 020; also at the contractor's facility f o r spacecraft 017 and
020 just before the integrated test.
The redundancy t e s t s revealed several malfunctions when the MCP was performing
the function correctly, but a p a r t of the redundancy capability was inoperative. During
the spacecraft 011 test, two of the redundant time delays were inoperative. When these
failures were detected, the spare MCP was installed and all the interfaces were reveri-
fied. (Refer to the section on time-delay failures f o r these failure analyses. )
42
HARDWARE PROBLEMS AND RESOLUTIONS
2. Providing a lot traceability at the supplier and u s e r levels for these qualified,
high- reliability components
Apollo environments
Relay Failures
Each MCP system used approximately 1050 relays to establish redundant switch-
ing logic. These hermetically sealed, microminiature armature relays were developed
in accordance with the product specifications and end- items specifications of the vendor
and had a single qualified source of supply. The reliability objective in the relay pro-
curement was the attainment of a life-failure-rate level of 0.08 percent in 10 000 relay
operations with a 90-percent confidence level and a maintained life- expectancy confi-
dence level of 60 percent p e r 100 000 operations at 303 K (85" F) under the specified
rated loads. A s shown in table VI, when a proper test and screening program is
established, the majority of the relay failures occurs before installation of the system
in the spacecraft. The last two columns of table VI are not exact because the same
MCP unit was flown on both spacecraft 017 and spacecraft 020 and, because the MCP
was not tested after the flight of spacecraft 020, relay failures could not be determined
to have existed. Some of the relay failures at the system- and spacecraft-test level
resulted from operator e r r o r s and were secondary- type failures caused by overcurrent
applications o r short circuits in the associated spacecraft wiring.
43
TABLE VI. - RELAY FAILURE HISTORY
Two relays were found to be inoperative during the spacecraft 011 postflight analy-
sis in September 1966. The subsequent failure analysis revealed solder- contamination
particles that caused a short circuit in the relay case. The source of these solder
particles was considered to be the soldering process to close the evacuation hole of the
relay; actually, a steel plug was snapped into the hole, and it was sealed by soldering.
The vendor terminated procurement of these solder-closeout relays on December 3,
1965. The new sealing process specified that the steel plug be snapped into the evacua-
tion hole and a ring spotweld be used to seal the hole. This new p r o c e s s was considered
to be contamination free. However, a total of 1843 of the relays procured before
December 3, 1965, had already been used in the various MCP flight systems. The iden-
tification of the specific MCP systems containing the solder- closeout relays was obtained
from the individual relay serial numbers and the lot traceability f o r high- reliability
p a r t s . System 1 f o r the MCP had 891 of the solder-closeout relays; system 2, 479;
system 3, 195; system 4, 167; system 5, 81; and system 6, 30.
After the locations of all the solder-closeout relays had been determined, these
relays were removed and replaced with welded-seal relays, but the task was not simple.
One technique was to X-ray the relays in a n attempt to determine whether the relays
contained solder particles. However, the results were unsatisfactory because no con-
clusive correlation could be made between the analyses of the X-rays and the actual
opening of the relay case and the physical inspection of the relay f o r contamination.
The relay X-ray technique was officially eliminated December 15, 1966. The delay in
delivery of such a large quantity of relays would result in a n unrealistic schedule if
each solder-closeout relay were to be replaced. On January 9, 1967, a rationale was
established f o r exchanging the relays. Circuits dealing with spacecraft recovery were
top priority, circuits dealing with mission success were second priority, and circuits
having more than one solder- closeout relay in any redundant path were third priority.
44
The vendor began replacing the relays in accordance with this rationale to meet the
spacecraft 017 scheduled delivery date of January.21, 1967. Control assemblies and
plug-in printed circuit boards were exchanged among the MCP systems to ensure that
correct relays were used in the MCP system f o r spacecraft 017. System 5 f o r the M C P
was delivered to the KSC on January 11, 1967. The scheduled events for spacecraft 017
at the KSC were delayed so that the allowed schedule relief could be used to incorporate
more improvements in the MCP. On February 9, 1967, the NASA and the spacecraft
contractor reviewed the solder- closeout- relay situation, and a mission- event- by-event
failure-effects analysis was used to a r r i v e a t a final determination f o r relay replace-
ment. The actual replacement process was difficult because the printed circuit track
of the control assemblies could be damaged while unsoldering the relay. The problem
was finally corrected when all the solder-closeout relays in critical circuits had been
replaced with welded- closeout relays.
Ti me - DeIay FaiIur e s
For a 14-month period ending in April 1967, NASA and various aerospace industry
representatives combined their efforts to resolve the M C P time-delay and fuse-diode
failures. The failure mode was readily established shortly after the first time-delay
circuit failed. Transistor Q (fig. 17) of the time-delay circuit shorted (collector to
1
base) causing zener diode CR to overload, exceed the thermal rating, and fail, s o that
1
45
28Vdc --- I I
A
--‘“I I
0 I .--
r---7 R1 I I
I
;f; -
a
28 V dc
B Typical
60052 load
I
I
I r I ,
r I
I
P Ground
Ground
Ill-$
@*%m
L_ _ _ _ _ _ _ _ _ _ _ _ _
w
8-,J
t
J
capacitance (total of all filters)
Frame of spacecraft command cofltroller
J
the fuse diode on the time-delay input was overloaded and failed (open circuit). Numer-
ous attempts were made to establish the cause of the failure. The following list of
significant events in the investigation indicates the amount of study on this problem.
3. On March 22, 1966, an unassociated failure occurred in the SCC, and the unit
was sent to the vendor f o r repair.
4. On March 24, 1966, the failures of two 3-second time-delay circuits in the
s a m e SCC was discovered during bench testing by the vendor before the repair and
rework.
5. On April 2, 1966, the MCP repair was completed by the vendor, and the unit
was reinstalled in spacecraft 011 to support testing. (At that time, no redundancy test
of the M C P had been performed by the spacecraft contractor.)
46
8. On August 10, 1966, during the flight readiness review, the representative
f o r the vendor explained that each time-delay network was checked in the MCP by the
bench test equipment before delivery, that a transient energy level of 250 V dc f o r
50 microseconds was required to break down the transistor, that the bench equipment
had been instrumented to search f o r transients, and that 43 V dc was the highest tran-
sient determined. (The transistors a r e rated for 80 V dc. ) Therefore, the representa-
tive concluded that the transient must be coming into the MCP from spacecraft wiring.
This assumption was reasonable because inverter failures and other high-voltage
spikes had previously been discussed at the flight readiness review. A recommendation
was approved to put four 18-V dc zener diodes across the MCP input direct-current
power buses in the spacecraft circuits external to the MCP.
9. On August 14, 1966, the zener diode network was approved f o r installation
in spacecraft 011.
10. On September 8, 1966, the zener diode modification was installed and opera-
tional in spacecraft 017.
11. In September 1966, 8-, 3-, and 0.5-second time-delay circuits were found to
have failed in SCC unit 5, which was supporting the spacecraft 017 tests. (These fail-
u r e s were found after the zener diode modification. )
12. In October 1966, the vendor continued to study the failures, attempted to
correlate the high- reliability lot traceability, and analyzed the testing procedures.
Two more time-delay failures were found in SCC unit 5.
13. On November 2, 1966, a special resistance check was devised to verify the
operation of 3- and 0. 2-second time-delay circuits at the KSC just before launch of
spacecraft 017.
14. In November 1966, the vendor's test equipment was modified to incorporate
the zener diode fix.
15. On December 12, 1966, a time-delay failure was found in SCC unit 6.
16. In December 1966, dual-redundant capacitors were added to the emi filters
to solve the polarized tantalum capacitor-failure problem discussed previously.
(During this investigation, the engineers discovered the large potential charge that
these filter capacitors could store. )
17. In January 1967, two more time-delay failures were found in SCC unit 6, and
a detailed extensive investigation was implemented.
18. In February 1967, the insulation-resistance megohm test used during accept-
ance testing was determined to be the cause of the failures. After the megohm test was
eliminated and a multimeter was used to check the insulation resistance, no more time-
delay failures occurred.
47
The extensive engineering investigation to resolve the time- delay failures revealed
that this failure, as well as several others, resulted from the transients caused by the
500-V dc megohm portion of the unit-level test at the vendor's facility. Failure of
8 diodes, 4 capacitors, 22 fuse diodes, and 14 time delays could be attributed directly
to these transients. The listed failures were of an induced nature. During factory test,
components were stressed beyond their capacity by an action of the test operator, who
was not aware of the consequences. The action consisted of shorting an internal
ground (GI, G2, o r G3) to the f r a m e of the SCC after the megohm test. (Refer to
points A to B in fig. 17. ) This action was taken by the test operator to avoid electrical
shock from filter capacitors that had been charged to several hundred volts when the
megohm test voltage was applied to the f r a m e of a control assembly in the SCC. The
control assemblies were insulated from the frame, and this ivsulation was tested each
time by the megohm test unit. The test specification required that 500 V dc be applied
a c r o s s a test point on the control assembly to the SCC frame f o r 2 minutes, then the
insulation resistance would be read. During this 2- minute period, the filter capacitors
were charged through the multitude of relay, time- delay, and differentiator paths.
Discussions with test personnel indicated that, instead of placing the megohm test
probe on the control assembly test point, it was more convenient to touch a screw near
the connector of the assembly. The screw was adjacent to exposed terminals that were
p a r t of the diode circuit. The probe could accidentally slip off the screw and contact
the terminals. When this happened, the time constant f o r charging the filter capacitors
was brief, and the capacitors charged up to slightly l e s s than 500 V dc. To eliminate
the possibility of electrical shock after completion of the insulation- resistance test,
the test operator would usually short an internal ground terminal (G1, G2, o r G3) to the
SCC frame, but sometimes the operator would short the metal plate of the control
assembly to the frame. At the instant the short was applied, certain components in the
command controller would be subjected to high electrical s t r e s s e s . At times, this
s t r e s s would exceed the breakdown strength of one o r more components and subsequently
resulted in a large transient current when the unshorted filter capacitors discharged.
48
transistor and zener diode had degraded o r shorted junctions that were essentially the
same as the failed parts in the SCC units of the MCP. During the laboratory test, the
lowest pulse input to cause failure was at 108 V dc (28 V dc plus an 80-volt pulse).
Six failures occurred at input signals of less than 150 V dc, 10 at less than 170, 20 at
l e s s than 198, and 35 at less than 228. Thirty of the transistors failed on the first
voltage pulse at the failure level, five transistors failed by the fifth pulse. The 21 tran-
s i s t o r s that did not fail were ultimately pulsed 170 times at 228 V dc. After the cause
of failure had been determined, it remained to be proved that the components that had
not failed were not degraded by the megohm tests, even though they seemed to function
satisfactorily. Three of the 21 transistors were selected at random and were pulsed
for a minimum of 43 000 cycles without a failure. Therefore, the misapplication of the
megohm test could cause immediate failures but would not cause the components to be
degraded and susceptible to subsequent failure.
On the basis of the vendor's successful isolation of the cause of the failures, its
proper corrective action to prevent future occurrences, and its demonstration that
existing time-delay transistors in the MCP were not degraded, the problem was closed
on April 6, 1967.
The problem with the MCP relays was a result of the vendor's fabrication tech-
niques, which required the leads to be clipped. The relays, including the pin seals,
were first inspected and then inserted in the circuit boards; the leads were clipped, the
cut pins were soldered, and the solder was inspected. (At this time, the relay-pin
seals could not be seen. ) The clipping operation mechanically s t r e s s e d the relay pins
and cracked some of the glass seals.
Whenever a glass seal cracked o r the relay pin became loose in the glass seal, the
pin was free to rotate; merely the wrist action of the technician clipping the pin could
rotate the pin and a l t e r the critical relay contact alinement (fig. 18). The manufacturing
alinement procedures required that the contacts first be set at a minimum
0.127-millimeter (0.005 inch) clearance. An overtravel adjustment was then checked
and established at 0.0635 millimeter (0.0025 inch) to effect the proper wiping action of
the relay contacts and then the proper electrical transfer. Alinement of the contacts
had to be within +3". A projected viewgraph, which magnifies the contacts many times
49
f Relay pins
Glass seal
Gold
Gold-plated contact
(b) Top view of relay pins with con-acts (c) Cross section of pin ent ring relay
in open configuration. header and the glass seal.
their actual size, was used by the manufacturer to inspect and check these adjustments.
Failure or improper action of the relays could occur for insufficient contact gaps o r for
contact gaps with too great a clearance. To prevent future problems, the vendor elimi-
nated the relay clipping during the manufacturing process. An NASA Flight Safety Infor-
mation Bulletin was published to notify all Apollo Program participants of the hazards
of clipping terminals with glass-to-metal seals.
FLIGHT PERFORMANCE
The MCP served the three final unmanned Apollo flights without a flight anomaly
caused by the system. The following discussion of the three spacecraft gives more
details concerning the flight performance.
Mission AS-202
Apollo mission AS- 202 (spacecraft O l l ) , using an uprated Saturn I launch vehicle,
was launched on February 26, 1966. The MCP consisted of SCC unit 4, GCC unit 1,
and ADS unit 1. The MCP system configuration production unit numbers are given to
emphasize that SCC unit 4 was refurbished to s e r v e as a spare for spacecraft 017 and
020. The contractor was officially notified on October 12, 1966, to perform this
refurbishment. The MCP was the first Apollo system to be considered f o r reuse in
flight.
50
The postlaunch report f o r mission AS-202 stated that the MCP "automated the
sequential event controls that would be normally operated by the flight crew. " The
report also stated that these sequential event controls, which included the MCP, 'Yunc-
tioned satisfactorily throughout the flight, and the related test objectives were met. I t
I
The only problem mentioned was a data-sampling-rate problem. Measurements
CE0321X and CE0322X (parachute disconnect at landing) were not received by
telemetry. The MCP disconnected the parachutes after landing but turned off the logic
bus power 35 to 40 milliseconds after the parachutes were disconnected. This sequence
was normal but it did not allow sufficient time f o r the telemetry, which had a sampling
rate of 10 samples p e r second (100 milliseconds between samples), to receive the sig-
nal before the bus power was removed.
Apollo 4 Mission
The Apollo 4 mission (spacecraft 017) was launched on November 9, 1967.
Apollo 4 was the first mission to use a Saturn V launch vehicle. The Apollo 4 Mission
Report stated that "sequencing of the mission control p r o g r a m e r was satisfactory
throughout the mission. The mission report continued as follows: "The mission con-
t r o l programer was primarily a passive device, and no specific instrumentation was
included f o r its analysis. Verification of continuity a t the proper time was the only
criterion considered during evaluation of this programer. P r o p e r performance was
indicated throughout the mission. '' The report could have included a statement that the
MCP was used f o r unmanned missions only and that the flight downdata link could not
provide measurements f o r the unmanned system; therefore, other interfacing system
measurements were evaluated to determine the MCP performance.
The postflight tests of the MCP, as stated in the mission report, evaluated the
MCP redundancy and verified that the programer had functioned properly. A more
detailed and complete inspection of the MCP was performed by the manufacturer during
the p r o c e s s of refurbishing the MCP f o r use as a s p a r e f o r spacecraft 020. This refur-
bishment was successfully accomplished because the s a m e MCP flown on spacecraft 017
was subsequently flown on spacecraft 020.
Apollo 6 Mission
The Apollo 6 mission (spacecraft 020) was launched on April 4, 1968. Apollo 6
was the second mission to use a Saturn V launch vehicle and was the last unmanned
Apollo mission. The Apollo 6 Mission Report stated that "sequencing of the mission
control p r o g r a m e r was satisfactory throughout the mission. '' A brief mention was made
concerning the stable I and stable I1 flotation attitudes. "The Apollo 6 mission was the
first mission i n which the command module assumed the stable I1 (inverted) flotation
attitude after landing. The ADS unit of the MCP correctly sensed this attitude and
initiated the sequence that uprighted the spacecraft.
The mission report continued: "The mission control programer supplied control
function inputs to various systems during the flight. No specific instrumentation was
designed t o analyze programer performance; however, verification of continuity at the
proper time showed proper programer performance throughout the mission. This
51
statement is true of all the MCP flight evaluations. No measurement points were avail-
able to determine the MCP outputs directly. The sequencing and performance of the
interfacing systems that were instrumented could be evaluated and thereby allow a n
indirect determination of the MCP performance.
The Apollo 6 report also stated that "the s a m e p r o g r a m e r (except f o r the altitude
and deceleration sensors) was used on the Apollo 4 and Apollo 6 missions. This MCP
had been refurbished to s e r v e as a spare f o r spacecraft 020, and a test anomaly at the
KSC resulted in the s p a r e MCP (which w a s previously flown on Apollo 4) being installed
in spacecraft 020. The NASA management decided to reuse this MCP after a thorough
analysis at the flight readiness review.
Unmanned Flight P l a n n i n g
The requirements f o r unmanned flights should be established early in the mission
planning to prevent changes from affecting the hardware design. The design of the
unmanned sequencing units should be as flexible as possible with the realization that,
in dynamic programs f o r space flights, several mission changes are inevitable.
The items and sequences that are subject to change during unmanned flights (such
as tape recorder sequencing, camera sequencing, gimbal actuator motor "off and 'I
52
be, as far as possible, relatively simple, flexible, and general purpose. Engineering
man-hours could have been better expended in actually testing the hardware manually
(for each of the six programer systems delivered) rather than in preparing automated
equipment to run the actual acceptance test in less time. The test equipment flexibility
should be such that it can be easily reconfigured and recertified f o r the numerous,
expected flight-hardware modifications.
Hardware Problems
The mission control programer relay failures can generally be attributed to a n
e a r l y method of solder sealing the evacuation and gas-filling hole. The solder sealing
process would sometimes result in solder particles inside the relay case; the particles
floated a c r o s s the contacts in the zero-g environment. A new sealing process was
developed in which a steel plug was snapped into the relay evacuation hole, and a ring
spotweld sealed the plug to the case. The new welding method ended the solder-
contamination problem. It is recommended that solder-sealed relays not be used on
future space flights.
The time- delay failures resulted in the most extensive failure analysis and inves-
tigation activity that occurred during the mission control programer development. One
area that was misleading during the failure analysis was that each time delay was
verified "good" during the programer contractor's acceptance test just before delivery
from the factory to the spacecraft; then, after the programer was installed in the vehicle
f o r the test support/interface verification activity, one to three time delays indicated
"failed" during the first spacecraft test of programer redundancy. The failure mode
was finally determined to be that the time-delay transistors failed "open circuit" by the
sudden discharge of a 500-volt potential that had been stored in the filter capacitors
during the megohm insulation test at acceptance, Only after much investigation was it
determined that the test operator shorted the filter capacitor to ground (not a normal
documented test procedure) just before delivery of the programer from the acceptance
test area. The discharge current followed several sneak paths and would burn out one
o r more of the time-delay transistors. The corrective action was to stop the megohm
insulation test as p a r t of acceptance. Future programs should ensure that test operators
53
be cautioned to follow the test procedures exactly and not add any unique procedure
such as manually discharging capacitor charge buildups as the unit is removed from the
teststand. After the 500-volt insulation check was eliminated, no transistor failures
occurred. Future electronic test designers should be especially aware that large filter
capacitors can s t o r e voltages of a sufficient level to damage low-voltage-level devices.