Saturday, June 8, 2019

Continuing the restoration of the Apollo Guidance Computer

RESTORING THE AGC

Checkout to be sure all modules are working properly

Finally we were in front of the Apollo Guidance Computer (AGC) again and could continue the restoration. First step was to check that all the prior parts of the system were still working properly. a task that took a day.

Main connector (360 pin A51)

The AGC is hooked to the rest of the spacecraft through a single 360 pin connector (A51) which routes signals between the computer and Lunar Module components like the Display Keyboard (DSKY), reaction control jets, rocket engines, control panels, gyroscopes and telemetry.

The AGC had been hermetically sealed and filled with nitrogen for most of time, almost five decades, that it sat exposed to the humidity of Houston. That left the bulk of the machine in pristine condition. The second smaller external test connector (A52) with its 144 pins had a sealed cover in place; this would be removed to allow technicians to hook up ground support equipment and test out the AGC.

Unfortunately, A51 had no cover and its 360 tiny female contacts were not protected. We inspected this carefully with a microscope and found that the pins were mostly straight, but had gunk that may be corrosion on them and lots of foreign material wedged into the holes.

Cleaning this requires tools like dentists use, but sharpened to be even smaller, plus cleaning instruments, applied laboriously one pin at a time. Since the pins are plated with 50 mils of gold, they weren't corroded, just coated with contaminants that could be removed. We are achieving about 10 cleaned pins per hour but can't work on the connector while the computer is together and being tested. It will be several more days before it is fully cleaned.

Checking out memory drive electronics

While in Houston last time, we didn't test the analog modules in the AGC that would drive access to the erasable (RAM) and fixed (ROM) memories. These consisted of modules such as Strand Select, Core Rope Drivers, Current Switch, Sense Amplifiers, and Erasable Memory Drivers.

We hadn't tested these last time because we had found that the erasable core memory module B12 had a fault that would cause bit 16 of every word to become a 1. We planned to repair B12 before continuing.

We successfully tested the Strand Select, Core Rope Driver and Sense Amplifier modules using copies of the original checkout procedures. The procedures were partial documents, so we had to study schematics and fill in the blanks.

Current Switch module debugging and repairs

When we got to the Current Switch module, however, we discovered two circuits inside with faults. The path to drive addressing current through a X or Y wire involves a special transformer core with four windings. These flip the core one way to drive current in the read direction, flip it another way to drive current in the write direction, and have the windings that output the current pulses in each direction.

One X wire and one Y wire circuit for addressing set of the transformers showed as open circuits, meaning that the computer couldn't address any words contained in 1/64th of the rows and 1/32nd of the columns, not just the one word that was at the coincidence of the failed X and Y circuits. 

Flight versions of the AGC had every electronic module and the backplane filled with some form of epoxy or other foam material to hold all parts in place during extreme accelerations and vibrations. These seal in the parts, making inspection and replacement very difficult.

Fortunately for us, only two modules were encased (potted is the common term used for this). Sadly, they were two broken modules - the core memory itself (B12) and the Current Switch module. While we had no feasible path to open and fix B12 due to the placement of the fault deep inside a sandwich of core planes, we could proceed on the newly discovered faulty unit.

Armed with mechanical drawings of our module from the NASA archives and pictures of an unpotted copy of the module located at the Computer History Museum, we knew exactly what lay under the potting and where it was to great accuracy.

Excavating the parts to inspect and repair was similar in some ways to excavating a dinosaur fossil from surrounding rock. We milled down through the epoxy until we found the first trace of white wire insulation. From that point, careful work with picks, tiny screwdrives, tiny chisels and a small hammer broke off small bits of the potting.

The encasing material was brittle enough to break without putting any damaging force on the components themselves. The result was a window through the potting, with the face of the cordwood module and its components exposed cleanly. Since these modules have parts that run through the cordwood from one face to the other, we needed to expose four windows in order to reach the two failed circuits inside.
First excavation to find and repair bad diode

Second excavation to find and repair the second bad diode
The serial number of the Current Switch module we worked on was not the same as the item shown in records to be installed in our AGC. Perhaps our machine was used as the donor for a working Current Switch, with the bad part stuck in our machine before it went to be sold off in a GSA auction.

The bad part in both circuits was a signal diode. It was a part that is still common today, 1N914B, so we drove to a local electronics shop and picked up some. The old diodes were detached and drilled out, our replacements slid into the module and soldered into circuit. Repaired!

Ground Support Equipment Monitor tested and put into use

Mike designed an FPGA based board to replicate the function of the three floor standing racks of ground support equipment used at MIT for checkout and debugging of the AGC. Samtec had delivered the new pins they manufactured to allow us to build or repair AGC connectors - this required creating all the tooling to accomplish a limited production run, quite expensive even with the original mechanical drawings of the pins.

This donation was greatly appreciated as it allowed us to build the 144 pin connector A52, put Mike's monitor board atop it and fit it to the AGC. Of course, we also had to make the special jacking screws that tightened the connector to the AGC and pulled it up and out of computer smoothly on removal. Marc had found screws that were close and then milled the remaining features to give us a new supply.

The newly created A52 connector with its attached monitor board was installed on the AGC and we turned on power to both. The design was good, all functions checked out well, so we could control and test the AGC from this point forward with it closed up and in its flight configuration.

A52 test connector with our ground support monitor being installed
The control panel of the ground support equipment was replaced by a graphical user interface that included a virtual DSKY as well as the hundreds of blinking lights, buttons and switches. It communicates over USB with the monitor board in A52.

The monitor can simulate both erasable and fixed memory along with all its other checkout functions. The erasable memory function needed a gate that was only added to AGCs after ours, but we did some wirewrap to enable that function, borrowing a spare gate inside the machine.

Mike could fire up Apollo software, such as Luminary 99 that flew on first moon landing. Initially, we had a Program alarm (0210) and certain virtual DSKY keys came out garbled. This was quickly fixed as we found two signals, one from the DSKY and one from the Intertial Measurement Unit (IMU) that needed to be set high. We wired up a jumper since the pins of A51 had still not been cleaned adequately.

Everything worked great. We ran through the entire self test suite of diagnostics using MIT written software such as Retread 44. The AGC logic was working flawlessly and, using the simulated fixed and erasable memory, we had a running computer.

Ground Support Equipment interface hooked to the A52 connector
In future days we need to complete cleaning of A51, build a replacement male connector and find a way to cable up signals to emulate parts of the spacecraft. My DSKY substitute also connects through 34 of the pins on A51, so it can't be wired into the machine and used until we have that connector completed. 

3 comments:

  1. Things are getting pretty exciting! Do you have photos to share of the Current Switch surgery?

    ReplyDelete
    Replies
    1. Yes, will amend the post with the picture in a few minutes

      Delete
  2. Thanks for the photos - I spent the morning at the dentist today and can sympathize.

    ReplyDelete