Another diode bites the dust in the AGC Current Switch Module

Setback with erasable memory, failure in Current Switch module

Near the end of our time with the AGC, we were running various portions of the Apollo missions such as LM powered descent initiation, LM rendezvous calculation to rejoin CM, or CM burn calculation to hit the earth entry interface.

Suddenly, we were seeing hardware restarts and the software was not running correctly. Using Mike's test monitor we quickly realized that erasable memory was not working properly. We could rapidly trace this down to a failure in the Current Switch module (again).

Testing diodes after powering down the computer led us to a third diode that failed open, just as the first two had been. We didn't have enough time left to mill out new windows in the polyurethane foam encapsulation, drill out the bad diode and insert a replacement.

This module has 28 such diodes - three have already failed and NASA documents show that Raytheon switched from the original diodes to new types due to failures similar to what we experienced. The original diodes had a mesa construction, something the semiconductor industry quickly moved beyond.

We kept the Current Switch Module here so we could remove all the encapsulation on the module, then replace the 26 original diodes with modern 1N914B diodes like we did with the first two we found. This should be completed so that we can rejoin the module with the AGC for live demonstrations we will do in July.

Back to Houston until our next session

The AGC flew back to its home in Houston, until our next session and subsequent demonstrations.

Final upgrade to our AGC and working on simulating missions such as lunar landing

Adding the PRO key support to this AGC

The original version of the block II computer had a key on the DSKY marked STBY that was used to put the computer into sleep mode (and bring it out again). Soon, it was replaced by the label PRO and was used for two purposes - standby same as before, and signaling PROceed when the astronaut was ready to approve some action proposed by the computer.

Prior to having a PRO key, the astronaut had to push VERB, push 3, push 3, and push ENTR to confirm a proposed action. When there are a sequence of requests from the program, such as during the start of a landing sequence, having to mash four keys to give assent took time. In some cases, the astronaut had just five seconds to approve the start of a burn; if the VERB 33 ENTR had a typo, it might be impossible to redo it fast enough.

The original STBY function is purely done in hardware. Once the software sets a bit that allows standby, the key itself was wired directly to circuitry that dropped the switchable 14V and 4V supplies, leaving only a base level of power for functions such as pulse generation needed by other hardware in the spacecraft.

The new second use of the key for approving action requires that the software be able to see when the key is pressed. This was accomplished by a small hardware change that routed the key signal from the standby circuitry to set a previously unused bit in one of the I/O channels. Our AGC, the last of the prototypes before the flight versions, did not have the PRO key wired to the channel yet.

We recreated the change, using wire wrap, such that when my DSKY has its PRO key pushed, it sets the bit in the channel to approve a program proposed action, but will still switch off the computer if held long enough to activate the standby circuit.

Challenges running the PDI program for the Apollo 11 landing

We have been able to run program 63, the way you accomplish Powered Descent Initiation (PDI) to lower the orbit of the LM and transition through to landing on the surface. However, once the program sets up for the burn of the descent engine and is given permission by the PRO key to fire, the lack of acceleration is detected. The software flashes Verb 97 which means thrust failure and we can't slow down or reach the surface.

Marc set up an accelerometer substitute using an ATMEL processor and three potentiometers, allowing us to indicate (crudely) that acceleration is occurring on any or all of the three axes. However, the question of which direction the acceleration should be injected is not a simple one. In fact, the answer necessary to bring the LM to the ground with zero forward velocity is devilishly hard to calculate.

The inertial measurement unit of the LM (and CM) consists of a platform (called the stable member) which has three gyroscopes and three accelerometers (PIPAs) mounted so that X, Y and Z rotation and acceleration can be measured.

This stable member is suspended on three concentric gimbals so that regardless of the rotation or movement of the spacecraft, the stable member will remain locked in the same position relative to the milky way galaxy. It drifts slightly over time, which is why it is aligned using the star sighting telescope to return it to its ideal position.

Each gimbal is used to read off rotation of the spacecraft around the stable member in one axis. For technical reasons, having only three gimbals means that the inner and outer gimbals can move so they are parallel to each other. This is bad because a rotation can't be assigned to only one axis, it instead is seen as a movement on both axes simultaneously. This foils the ability to recognize movement properly and is called Gimbal Lock.

Certain rotations of the spacecraft can move it so that it approaches or reaches Gimbal Lock. Once in gimbal lock, the stable member must be reinitialized and star sightings are necessary to align it, a time consuming process. To avoid this, the people in Flight Planning (and in Mission Control) have to plan out the orientation of the stable member so that all the planned maneuvers of the spacecraft can be made without approaching gimbal lock.

Consider the way the spacecraft evens out heat and cold as it flies between earth and moon - the Passive Thermal Control mode - where the spacecraft has to spin on an axis like food on a barbecue spit, with the axis at right angles to the sun. Turning one rotation every two minutes means the stable member is rotating on its gimbals at the same rate and that motion can't be allowed to cause gimbal lock.

At many times during the mission, the orientation of the stable member relative to the galactic background is changed to put it in the best position for the maneuvers ahead. The orientation information is sent up as a REFSSMMAT, a reference stable member matrix either vocally or by the ground using 'remote control' to press DSKY keys to enter the new data in the AGC.

The flight director/attitude indicator display (the 8 ball) is tied to the stable member, thus it is showing where the stable member is sitting while the spacecraft rotates around it. That works out find when in a straight line trajectory such as during translunar coasting, but introduces problems when in orbit around the earth or the moon.

Since the stable member remains fixed relative to the background of the universe, the orbital path will cause the member to appear to rotate as the spacecraft circles the globe. If we start with the stable member parallel to the moon's surface at one point and leave the spacecraft in that same position, as we orbit our nose will appear to pitch up then nose over until we seem to be upside down relative to the moon when we are at the opposite point of the orbit, gradually rotating again until we are flat and parallel at the starting point of the orbit.

The flight director/attitude indicator will rotate around with the stable member (and spacecraft). If we cause the spacecraft to pitch at exactly the right rate, then the spacecraft 'floor' will always be pointing down towards the moon. Our stable member, as a consequence, will be rotating as we orbit and our 8-ball won't make sense.

A special device named ORDEAL will permute the 8-ball so that it seems to match the 'floor' of our spacecraft, and cause the correct pitch rate, so that the pilot does not see the stable member rotating. However, regardless of the tweaked display, the reality of an orbit means that the stable member is rotating while we circle the moon.

That means that the accelerometers idea of X, Y and Z is based on the stable member and are rotating compared to our spacecraft. During the landing, therefore, the direction of the thrust vector of the descent engine is rotating and has to be calculated based on the original setting of the REFSSMMAT, the place we are in the lunar orbit and the orientation of the LM compared to the moon's surface.

The landing of Apollo 11 begins with the crew oriented to look down at the moon's surface with the descent engine bell pointed in the direction of the orbit. Later in the descent, the LM pitches around so that the astronaut's feet are pointed down at the surface and they are looking forward in the direction they are moving. All these occur while the stable member stays fixed in inertial space.

As you can now see, the direction of acceleration we have to input to the AGC, which are given strictly to the X, Y and Z direction of the stable member, have to be manipulated based on all the factors above.

One simplification is that we don't care how the LM moves from face down to face forward since the stable member doesn't change at all. What does matter, however, is the effect of that movement on the thrust vector relative to the 1/6 G acceleration of the moon on the spacecraft. That must be factored in, both by the AGC in computing the burn and pitch up timing, but also in our injected acceleration.

We will need to issue AGC commands to look at the orientation of the stable member at the time we begin the engine burn, calculate how this rotates as we move around the orbital path/descent path, and apply acceleration appropriate to that.

So much more complicated that simply delivering input on the axis of the descent engine bell of the LM. We expect we will even see an alarm thrown after we think we have landed, unless we provide acceleration at 1/6 G in the proper direction to match where the moon surface is relative to the stable member.

Interacting with the running Apollo Guidance Computer using DSKY

Testing the DSKY substitute

We hooked up the DSKY substitute to the Apollo Guidance Computer and verified that all the lights, displays and keys worked properly. We now have a physical manifestation of the interface used by the astronauts to interact with the computer.

Video of DSKY working with AGC

Testing Ben Krasnow's electroluminescent panel

Ben (Applied Science YouTube channel) came over with the EL panel he built to NASA specs and we worked to interface it to my DSKY substitute. After a bit of work, I got it mirroring the display on my DSKY as the AGC ran various Apollo era software.

Video of Applied Science electroluminescent panel used with AGC

It was great to see a real electroluminescent display providing the output from the computer. Realism of the interaction took a step up.

Using the DSKY with the AGC

We ran several portions of the Apollo 11 LM mission, using the DSKY to interact with the Luminary 99 flight software that flew in the spacecraft. This included calculating the rendezvous burn after ascent from the lunar surface and of course the powered descent down to the surface. No 1202 alarms for us, because we didn't have a simulated Rendezvous Radar feeding pulses to overload the computer.

Erasable (core memory RAM) of Apollo Guidance Computer repaired and working properly

New flaw in Current Switch module

After having replaced the two failed diodes in this module, we began testing it with pulses to ensure that it worked properly at the currents and timing needed to drive core memory. This module consists of a number of large ferrite cores with four sets of windings on each. One winding is tied to all the cores and is used to switch them all back to the 'off' magnetic orientation. A second winding is used to select a core by switching it to the 'on' orientation.

Certain cores are selected by flipping them on. This flip of magnetic state induces a pulse in the third wiring, which causes a transistor to conduct to drive current through an X or Y addressing line in the B12 core memory module. Later, when all the cores are reset by the first winding, only the ones that had been selected will flip back. This induces a pulse in the fourth winding, turning on a transistor to drive current in the opposite direction through the X or Y addressing line.

This scheme is clever because the core retains the selected address from when it is selected. This selection induced the pulse to read out a word of the B12 module but also remembers which address lines were selected. The AGC logic does not have to retain the address in a register until later in the memory cycle when the word of erasable memory is rewritten - the selection cores themselves hold that information.

One one of the cores, we had a 4 ohm short in some module I/O pins that shouldn't be connected at all. After excavating the potting material around the suspicious area, we quickly excluded simple to repair causes such as shorting wires, shorted interconnect, etc. We found that the first and fourth windings were somehow shorting inside the core - these are the reset winding and the winding that produces a pulse when a previously selected core is reset.

Shorted windings in B11 Current Switch Module circuit

Careful examination showed us that it would be impossible to extract the core/windings from inside the cordwood module where they were epoxied in place. At least, impossible to remove it without severe damage to that core and its windings.

We have a suitable replacement core but putting four windings of 50, 32, 32 and 20 turns would be extremely challenging given the small diameter of the core. Fortunately, Marc and Mike figured out a clever hack to give us an equivalent functionality. It began by completely disconnecting the fourth winding, the one that switches on a transistor when a selected core is reset. That effectively cured the short since the shorted winding was no longer connected to anything.

Then, we installed a transistor of opposite polarity tied to the third winding, the one that is normally used only while the core is selected. When the selection pulse flips the core, the third winding sees a positive direction pulse, causing the attached NPN transistor to conduct. When the core is later reset, the pulse is in the negative direction thus the transistor doesn't turn on.

Yet, if we hook that third winding to a PNP transistor as well as to its NPN transistor, the pulses from the windings will cause one transistor to turn on with a positive going pulse, and the other transistor to turn on with the negative going pulse. This produces the same behavior as the original circuit, but requires only three windings instead of four. That is fortunate because we don't have four useful windings.

We inserted a small PNP transistor and did some rewiring, which gave us a module that passed all tests with flying colors. We were then ready to install the erasable memory driver modules, this current switch module, and our erasable memory (core stack) into the AGC.

PNP transistor wired to third winding of circuit

Archiving prior contents of the B12 erasable memory

We still have the flaw in the core memory stack - the inhibit wire for one of the data bits (bit 16) is an open circuit. In the read portion of a memory cycle, the selected X and Y address lines flip all cores of that word to zero. Any of the bits in that word which had a 1 stored in it will flip, causing a pulse to be detected by the sense amplifiers. This is how core memory does a read, by destructively flipping the word to zero.

During the reset pulse from the current switch module core circuits, the cores for the selected X and Y address lines, those whose current switch cores had been set on, will flip all the bits of the word to a 1 state. However, we don't want them to be 1, as some should be 0. That is the purpose of the inhibit wire - a signal on the inhibit wire will block the reset from flipping that particular bit to 1.

Reading data involves erasing it first, then rewriting the original value (or putting a new value in if this is a write operation). With a bad inhibit wire, bit 16 will be rewritten (or written) to 1 regardless of our desired value. Thus the loss of the inhibit wire renders that bit useless throughout all 2K words of erasable memory.

If the inhibit wire damaged happened after the computer was last powered down, then it hadn't yet forced a 1 into every bit 16 in the core stack. We had the opportunity to read the contents of the core memory correctly, since the read portion of a memory access doesn't use the inhibit wire. However, this is a one shot opportunity, as the process of reading any word will jam in the 1 in bit 16 during rewrite.

We powered up with memory access blocked through the test connector, put the machine in single instruction mode, and tested by reading a few memory addresses. We did find that bit 16 sometimes had the value 0, sometimes 1, verifying that the damage to the module was not present when it was being accessed 40-50 years ago.

We wrote down the values we saw in the test locations, then used Mike's test monitor to run through all of memory and retrieve the values. Folding in the few manual reads gave us a complete file of the prior contents of our memory module B12.

Mike studied the contents and could confirm that this was running a version of Aurora software, similar but not identical to the release for which there is an archived image. He could decode the contents of the DSKY display. It showed that the last command entered was a coarse alignment of the IMU (gyroscopes and accelerometers) to angles of 0, 0, and 90. The Gimbal Lock warning light was on, which may have been the event that caused the operator to perform a coarse alignment. Finally, the code stores a latitude when aligning the gyros - it was the location of the Johnson Spacecraft Center in Houston.

Rewiring to swap parity and data bits in B12 core memory module

Now that the prior contents are safely archived, we can implement our modification to make the core erasable memory functional again. Our great fortune is that the erasable memory module contains an error detection method called parity, to capture the cases where some radiation event might have flipped a data bit randomly.

This works by adding a sixteenth bit to each word - data bits are 0 to 14 plus 16, with bit 15 representing parity. The rule for parity is that the number of bits in a word which are 1 has to be an odd quantity. If not, the parity bit is set to 1 thus making the entire word have an odd count. if the data bits themselves have an odd count of 1s, the parity bit is set to 0.

At the end of a read from a word of core memory, the processor counts the 1 bits, determines if parity should have been 1 or 0 and then compares it to the parity bit read from memory. Mismatches raise an erasable memory parity alarm.

The erasable memory is quite reliable, especially since our AGC is not out in space subject to radiation events. We therefore don't really need parity checking. That gives us a working bit 15 which we can substitute for the broken data bit 16. As long as we can disable the parity checking, blocking the alarm, we will have a working memory.

Some backplane wiring was introduced to swap bits 15 and 16, as well as block the signal that performs a parity check. With the wirewrap changes completed, we closed up the AGC and tested. Success! The erasable memory is fully functional and we can run software fully out of the repaired erasable memory and the core rope simulator boxes providing the fixed (core rope) read only memory, just as the machine

Main spacecraft connector and patch panel finished, accelerometer simulator plugged in

Main connector

All connections between the spacecraft and the Apollo Guidance Computer take place through the main connector (A51). This has 360 pins, six rows high and 60 columns wide. The DSKY itself connects through A51, which made this a priority to implement.

As I detailed in an earlier post, this is a labor intensive cleaning process but finally we had all of them ready for a male connector to be inserted. We had designed an aluminum plate to hold the 360 Malco mini-wasp pins and to position a printed circuit board we designed atop it. The pins were soldered to the PCB and the far end held eight HD50 connectors (familiar to those who used these as SCSI cables).

Connector attached to AGC carrying 360 signals out for our access

The connector was inserted with a partial population of pins - those we needed for our demonstrations. We will finish installing the remaining pins later. The eight cables were run from the PCB out to the patch panels that gave us the same six row by 60 column array as banana plug receptacles. We used this to hook up the DSKY and simulated switches and other spacecraft systems.

Accelerometer simulator developed

Marc designed an Arduino based module that would simulate the accelerometers of the LM spacecraft. These Pendulous Integrating Pulse Accelerometers (PIPAs) swing a small pendulum back and forth kicked by three pulses in one direction followed by three in the other. When no external acceleration exists, the PIPA outputs a steady stream of three positive then three negative pulses.

PIPA simulator plugged into patch panel

The AGC software checks for this pattern and throws up a Program alarm (212) if they are not present, since that means the inertial navigation and digital autopilot functions can't be performed. Marc's device would deliver this pulse stream into the appropriate pins of the A51 connector, through our patch panel. He also designed it to allow him to add extra pulses in either direction to reflect acceleration of the LM.

Retrieving previously lost Apollo programs from a museum's core rope modules

Archiving

We visited the Computer History Museum which has a pair of core rope modules containing the Retread 50 software. These are part of their exhibition of an Apollo Guidance Computer, but we arranged to archive the contents of the modules using our working AGC, in support of their software history effort and to deepen understanding of what is in their artifacts.

Before we connected these to our AGC, it was important to verify that there were no open circuits on the strand and module select lines or other inputs as our computer could be damaged if that was the case. Following a long test plan, we determined that the rope modules were safe to insert.

We first started the AGC in single instruction mode, turned on the core rope access and tried a couple of accesses to see if they appeared to be delivering valid contents, based on the known archive of the predecessor Retread 44 software. It matched.

Mike then used his test monitor tool to read through all of the core rope modules and save their contents. We did spot some parity errors which were confine to specific parts of the modules. In fact, all errors were in the first (B1) module and they were in strands 1 and 9. Modules have 12 strands each holding 512 words of data (thus 6K capacity per module).

For strand 1, a particular data bit was always reading as a 1 value. This appears to be from a failed component inside the core rope module. We did not open the modules as they are artifacts that should not be disturbed. When a word legitimately had a 1 value in that bit, no parity error ensued, but every location that intended a 0 value would trigger a parity alarm.

Strand 9 had more erratic results, but again for just a single bit location in every word. That meant it too could be recovered by looking at the parity bit value.

We wrote a quick post-processing routine to use the parity bit to interplate the correct value of our bad bit. In early programs such as Retread, they left any unused words unwired - thus they would legitimately produce a parity error with all zero results. We had to find those unused words since we had improperly corrected it to have a non-zero word value.

With the data recovered, it could be shared and analyzed. Many of the expected changes were there - since Retread 44 was written on the emulator before the first block II hardware was built, it had some code that failed on real hardware due to inaccuracies in the emulator. All of these flaws in code were corrected, many as were expected but some through alternate changes.

There is some added code, parts that are seen in later (Aurora) software but some that are unique to this previously lost software version. Analysis continues

We hope to archive other lost software versions as owners of other rope modules permit us to plug them in to our system.

Finishing equipment to read core rope programs and drive the AGC with real Apollo software

Core Rope Simulator box debugging

The core rope simulator boxes that came with the Apollo Guidance Computer were built by Raytheon and designed to cable up to a large console that would host the ROM program in rewritable (RAM) storage. We only had the two boxes inserted in place of the AGC core rope modules. not the large console nor the cables.

After Ken had reverse engineered the boxes and developed a Beaglebone-based driver to serve up the ROM programs to the boxes, we set into the process of debugging. First up, we had to find and repair about a dozen faults, many of them broken wires, as well as reshape IC leads to work in the very unreliable DipStik connectors used in the box.

After the Raytheon hardware was working it was time to test the function of Ken's driver box. It mostly worked right out of the chute, but we had to chase down some defects that caused errors reading certain blocks of memory.

The core rope modules that are normally used with the AGC consist of six separate modules which represent six blocks of 6K words. We will call them R1, R2, S1, S2, T1 and T2. The lowest addresses are in R1 and the highest contained in T2. The faults we experienced were when accessing S2, T1 and T2.

The Raytheon box would latch in the request for a particular module, then which of the 512 cores in that module should be read and which of 12 words from that module are passed into the computer sense amplifiers. What we found was that bad addresses were being latched due to transient short pulses that were due to poor logic design, a race hazard.

We could see the glitch, a pulse on the order of 100 ns compared to the real pulses that occur later and are microseconds in duration. The particular way that the addresses were decoded inside the Raytheon boxes allowed these to pass to the trigger that set flipflops, even though some of the address information had not yet made it through the other gates.

The solution was to add a small capacitor to the trigger line, such that it would ignore pulses that were too short but still lock in addresses when the desired signals arrived. The time constant that would block the glitch kept the integrated very short signal from climbing high enough to trigger the flipflop.

When we inserted a capacitor that was about 100 times too large, it began to delay the real signals - as their leading edge was spread out over a long slope. This caused some other addressing issues, but a capacitor that was just a couple times larger than the glitch did the job perfectly.

These boxes were serial number 3, but it is hard to see how they could have worked correctly on an Apollo Guidance Computer. That is, when the ROM was more than half full. They might have sufficed for small programs that remained in sections R1, R2 and S1. In any case, the core rope simulator works just fine now, letting us run real software such as Luminary 99 that was installed in the LM for the Apollo 11 mission.

Core Rope Module jumpers designed and manufactured

We intend to read existing core rope modules from both museums and private collectors, but a few early sets don't use all six rope modules. The code can fit in just two for Retread 50, as an example, thus four slots are empty. To work properly, jumper plugs must be installed in the four unused slots.

We didn't have any jumpers, but we could work out what was needed. Mike designed some connector housings, built them using a 3D printer, and populated them with the Samtec built Malco Mini-Wasp connectors. He installed small circuit board to complete the jumper circuit, and added handles for insertion and removal. These worked fine, setting us up for the opportunities ahead to read and archive the contents of core rope modules.

Rope jumper connector in place

Connector removal using handle

Continuing the restoration of the Apollo Guidance Computer

RESTORING THE AGC

Checkout to be sure all modules are working properly

Finally we were in front of the Apollo Guidance Computer (AGC) again and could continue the restoration. First step was to check that all the prior parts of the system were still working properly. a task that took a day.

Main connector (360 pin A51)

The AGC is hooked to the rest of the spacecraft through a single 360 pin connector (A51) which routes signals between the computer and Lunar Module components like the Display Keyboard (DSKY), reaction control jets, rocket engines, control panels, gyroscopes and telemetry.

The AGC had been hermetically sealed and filled with nitrogen for most of time, almost five decades, that it sat exposed to the humidity of Houston. That left the bulk of the machine in pristine condition. The second smaller external test connector (A52) with its 144 pins had a sealed cover in place; this would be removed to allow technicians to hook up ground support equipment and test out the AGC.

Unfortunately, A51 had no cover and its 360 tiny female contacts were not protected. We inspected this carefully with a microscope and found that the pins were mostly straight, but had gunk that may be corrosion on them and lots of foreign material wedged into the holes.

Cleaning this requires tools like dentists use, but sharpened to be even smaller, plus cleaning instruments, applied laboriously one pin at a time. Since the pins are plated with 50 mils of gold, they weren't corroded, just coated with contaminants that could be removed. We are achieving about 10 cleaned pins per hour but can't work on the connector while the computer is together and being tested. It will be several more days before it is fully cleaned.

Checking out memory drive electronics

While in Houston last time, we didn't test the analog modules in the AGC that would drive access to the erasable (RAM) and fixed (ROM) memories. These consisted of modules such as Strand Select, Core Rope Drivers, Current Switch, Sense Amplifiers, and Erasable Memory Drivers.

We hadn't tested these last time because we had found that the erasable core memory module B12 had a fault that would cause bit 16 of every word to become a 1. We planned to repair B12 before continuing.

We successfully tested the Strand Select, Core Rope Driver and Sense Amplifier modules using copies of the original checkout procedures. The procedures were partial documents, so we had to study schematics and fill in the blanks.

Current Switch module debugging and repairs

When we got to the Current Switch module, however, we discovered two circuits inside with faults. The path to drive addressing current through a X or Y wire involves a special transformer core with four windings. These flip the core one way to drive current in the read direction, flip it another way to drive current in the write direction, and have the windings that output the current pulses in each direction.

One X wire and one Y wire circuit for addressing set of the transformers showed as open circuits, meaning that the computer couldn't address any words contained in 1/64th of the rows and 1/32nd of the columns, not just the one word that was at the coincidence of the failed X and Y circuits. 

Flight versions of the AGC had every electronic module and the backplane filled with some form of epoxy or other foam material to hold all parts in place during extreme accelerations and vibrations. These seal in the parts, making inspection and replacement very difficult.

Fortunately for us, only two modules were encased (potted is the common term used for this). Sadly, they were two broken modules - the core memory itself (B12) and the Current Switch module. While we had no feasible path to open and fix B12 due to the placement of the fault deep inside a sandwich of core planes, we could proceed on the newly discovered faulty unit.

Armed with mechanical drawings of our module from the NASA archives and pictures of an unpotted copy of the module located at the Computer History Museum, we knew exactly what lay under the potting and where it was to great accuracy.

Excavating the parts to inspect and repair was similar in some ways to excavating a dinosaur fossil from surrounding rock. We milled down through the epoxy until we found the first trace of white wire insulation. From that point, careful work with picks, tiny screwdrives, tiny chisels and a small hammer broke off small bits of the potting.

The encasing material was brittle enough to break without putting any damaging force on the components themselves. The result was a window through the potting, with the face of the cordwood module and its components exposed cleanly. Since these modules have parts that run through the cordwood from one face to the other, we needed to expose four windows in order to reach the two failed circuits inside.

First excavation to find and repair bad diode

Second excavation to find and repair the second bad diode

The serial number of the Current Switch module we worked on was not the same as the item shown in records to be installed in our AGC. Perhaps our machine was used as the donor for a working Current Switch, with the bad part stuck in our machine before it went to be sold off in a GSA auction.

The bad part in both circuits was a signal diode. It was a part that is still common today, 1N914B, so we drove to a local electronics shop and picked up some. The old diodes were detached and drilled out, our replacements slid into the module and soldered into circuit. Repaired!

Ground Support Equipment Monitor tested and put into use

Mike designed an FPGA based board to replicate the function of the three floor standing racks of ground support equipment used at MIT for checkout and debugging of the AGC. Samtec had delivered the new pins they manufactured to allow us to build or repair AGC connectors - this required creating all the tooling to accomplish a limited production run, quite expensive even with the original mechanical drawings of the pins.

This donation was greatly appreciated as it allowed us to build the 144 pin connector A52, put Mike's monitor board atop it and fit it to the AGC. Of course, we also had to make the special jacking screws that tightened the connector to the AGC and pulled it up and out of computer smoothly on removal. Marc had found screws that were close and then milled the remaining features to give us a new supply.

The newly created A52 connector with its attached monitor board was installed on the AGC and we turned on power to both. The design was good, all functions checked out well, so we could control and test the AGC from this point forward with it closed up and in its flight configuration.

A52 test connector with our ground support monitor being installed

The control panel of the ground support equipment was replaced by a graphical user interface that included a virtual DSKY as well as the hundreds of blinking lights, buttons and switches. It communicates over USB with the monitor board in A52.

The monitor can simulate both erasable and fixed memory along with all its other checkout functions. The erasable memory function needed a gate that was only added to AGCs after ours, but we did some wirewrap to enable that function, borrowing a spare gate inside the machine.

Mike could fire up Apollo software, such as Luminary 99 that flew on first moon landing. Initially, we had a Program alarm (0210) and certain virtual DSKY keys came out garbled. This was quickly fixed as we found two signals, one from the DSKY and one from the Intertial Measurement Unit (IMU) that needed to be set high. We wired up a jumper since the pins of A51 had still not been cleaned adequately.

Everything worked great. We ran through the entire self test suite of diagnostics using MIT written software such as Retread 44. The AGC logic was working flawlessly and, using the simulated fixed and erasable memory, we had a running computer.

Ground Support Equipment interface hooked to the A52 connector

In future days we need to complete cleaning of A51, build a replacement male connector and find a way to cable up signals to emulate parts of the spacecraft. My DSKY substitute also connects through 34 of the pins on A51, so it can't be wired into the machine and used until we have that connector completed. 

The DSKY substitute is completed and ready to attach to the AGC

COMPLETING THE DSKY

Assembly and mechanical improvements

The issues discovered with the keyboard were caused by the shape of the cardboard shims that guide the keys up and down in their channels. Rob Lion worked on those and on the mechanical assembly of the keyboard and the display parts to the case.

All that remained was to drill the holes to attach the acrylic base to the aluminum enclosure, to fasten the faceplate on, and to do some final gluing of loose parts. One faceplate panel was sliding in its cover and keycaps were coming off because the prior glue was not adequate. The DSKY substitute is now sitting ready to be wired to the Apollo Guidance Computer.

Wrapping up the acceptance testing

The remaining series of DSKY tests after the last post were:

1. set up the register value (M12-M15) for lights and toggle each of the remaining
1. Vel 
2. Alt
3. No DAP
4. Prog
5. No Att
6. Prio Disp
2. Set up register value and toggle signs
3. Set digits in Prog display
4. Set digits in Verb display
5. Set digits in Noun display
6. drop 14, verify they turn off, turn 14 on, verify they are blank
7. Set up Verb, Noun and Prog, then turn on VNflash and observe
8. Oscillate VNflash
9. set digits in registers
1. R1
2. R2
3. R3
10. verify split register changes both R characters
11. flash key rel light
12. verify Pro key detected
13. verify keypress release works (on while key is depressed)
14. verify reset key discrete
15. verify all key codes sent

I completed everything down to step 12, trying every character value and all positions. I had to make some changes to my code to properly handle the VNflash discrete. That cycles at 1.5Hz and when it is on, the Verb and Noun are supposed to blank out. It causes them to blink, in other words. My initial spot in the code wasn't always getting executed, but I fixed that quickly.

Testing the keypresses is even easier - no need for 14 or 28 volt supplies, just the 5V line into the DSKY and hook the eight output wires up to eight pins on an Arduino. I discovered some mechanical issues with the keys that needed to be addressed but electrically they were perfect.