Setback with erasable memory, failure in Current Switch module
Near the end of our time with the AGC, we were running various portions of the Apollo missions such as LM powered descent initiation, LM rendezvous calculation to rejoin CM, or CM burn calculation to hit the earth entry interface.
Suddenly, we were seeing hardware restarts and the software was not running correctly. Using Mike's test monitor we quickly realized that erasable memory was not working properly. We could rapidly trace this down to a failure in the Current Switch module (again).
Testing diodes after powering down the computer led us to a third diode that failed open, just as the first two had been. We didn't have enough time left to mill out new windows in the polyurethane foam encapsulation, drill out the bad diode and insert a replacement.
This module has 28 such diodes - three have already failed and NASA documents show that Raytheon switched from the original diodes to new types due to failures similar to what we experienced. The original diodes had a mesa construction, something the semiconductor industry quickly moved beyond.
We kept the Current Switch Module here so we could remove all the encapsulation on the module, then replace the 26 original diodes with modern 1N914B diodes like we did with the first two we found. This should be completed so that we can rejoin the module with the AGC for live demonstrations we will do in July.
Back to Houston until our next session
The AGC flew back to its home in Houston, until our next session and subsequent demonstrations.
Tuesday, June 25, 2019
Monday, June 24, 2019
Final upgrade to our AGC and working on simulating missions such as lunar landing
Adding the PRO key support to this AGC
The original version of the block II computer had a key on the DSKY marked STBY that was used to put the computer into sleep mode (and bring it out again). Soon, it was replaced by the label PRO and was used for two purposes - standby same as before, and signaling PROceed when the astronaut was ready to approve some action proposed by the computer.
Prior to having a PRO key, the astronaut had to push VERB, push 3, push 3, and push ENTR to confirm a proposed action. When there are a sequence of requests from the program, such as during the start of a landing sequence, having to mash four keys to give assent took time. In some cases, the astronaut had just five seconds to approve the start of a burn; if the VERB 33 ENTR had a typo, it might be impossible to redo it fast enough.
The original STBY function is purely done in hardware. Once the software sets a bit that allows standby, the key itself was wired directly to circuitry that dropped the switchable 14V and 4V supplies, leaving only a base level of power for functions such as pulse generation needed by other hardware in the spacecraft.
The new second use of the key for approving action requires that the software be able to see when the key is pressed. This was accomplished by a small hardware change that routed the key signal from the standby circuitry to set a previously unused bit in one of the I/O channels. Our AGC, the last of the prototypes before the flight versions, did not have the PRO key wired to the channel yet.
We recreated the change, using wire wrap, such that when my DSKY has its PRO key pushed, it sets the bit in the channel to approve a program proposed action, but will still switch off the computer if held long enough to activate the standby circuit.
Challenges running the PDI program for the Apollo 11 landing
We have been able to run program 63, the way you accomplish Powered Descent Initiation (PDI) to lower the orbit of the LM and transition through to landing on the surface. However, once the program sets up for the burn of the descent engine and is given permission by the PRO key to fire, the lack of acceleration is detected. The software flashes Verb 97 which means thrust failure and we can't slow down or reach the surface.
Marc set up an accelerometer substitute using an ATMEL processor and three potentiometers, allowing us to indicate (crudely) that acceleration is occurring on any or all of the three axes. However, the question of which direction the acceleration should be injected is not a simple one. In fact, the answer necessary to bring the LM to the ground with zero forward velocity is devilishly hard to calculate.
The inertial measurement unit of the LM (and CM) consists of a platform (called the stable member) which has three gyroscopes and three accelerometers (PIPAs) mounted so that X, Y and Z rotation and acceleration can be measured.
This stable member is suspended on three concentric gimbals so that regardless of the rotation or movement of the spacecraft, the stable member will remain locked in the same position relative to the milky way galaxy. It drifts slightly over time, which is why it is aligned using the star sighting telescope to return it to its ideal position.
Each gimbal is used to read off rotation of the spacecraft around the stable member in one axis. For technical reasons, having only three gimbals means that the inner and outer gimbals can move so they are parallel to each other. This is bad because a rotation can't be assigned to only one axis, it instead is seen as a movement on both axes simultaneously. This foils the ability to recognize movement properly and is called Gimbal Lock.
Certain rotations of the spacecraft can move it so that it approaches or reaches Gimbal Lock. Once in gimbal lock, the stable member must be reinitialized and star sightings are necessary to align it, a time consuming process. To avoid this, the people in Flight Planning (and in Mission Control) have to plan out the orientation of the stable member so that all the planned maneuvers of the spacecraft can be made without approaching gimbal lock.
Consider the way the spacecraft evens out heat and cold as it flies between earth and moon - the Passive Thermal Control mode - where the spacecraft has to spin on an axis like food on a barbecue spit, with the axis at right angles to the sun. Turning one rotation every two minutes means the stable member is rotating on its gimbals at the same rate and that motion can't be allowed to cause gimbal lock.
At many times during the mission, the orientation of the stable member relative to the galactic background is changed to put it in the best position for the maneuvers ahead. The orientation information is sent up as a REFSSMMAT, a reference stable member matrix either vocally or by the ground using 'remote control' to press DSKY keys to enter the new data in the AGC.
The flight director/attitude indicator display (the 8 ball) is tied to the stable member, thus it is showing where the stable member is sitting while the spacecraft rotates around it. That works out find when in a straight line trajectory such as during translunar coasting, but introduces problems when in orbit around the earth or the moon.
Since the stable member remains fixed relative to the background of the universe, the orbital path will cause the member to appear to rotate as the spacecraft circles the globe. If we start with the stable member parallel to the moon's surface at one point and leave the spacecraft in that same position, as we orbit our nose will appear to pitch up then nose over until we seem to be upside down relative to the moon when we are at the opposite point of the orbit, gradually rotating again until we are flat and parallel at the starting point of the orbit.
The flight director/attitude indicator will rotate around with the stable member (and spacecraft). If we cause the spacecraft to pitch at exactly the right rate, then the spacecraft 'floor' will always be pointing down towards the moon. Our stable member, as a consequence, will be rotating as we orbit and our 8-ball won't make sense.
A special device named ORDEAL will permute the 8-ball so that it seems to match the 'floor' of our spacecraft, and cause the correct pitch rate, so that the pilot does not see the stable member rotating. However, regardless of the tweaked display, the reality of an orbit means that the stable member is rotating while we circle the moon.
That means that the accelerometers idea of X, Y and Z is based on the stable member and are rotating compared to our spacecraft. During the landing, therefore, the direction of the thrust vector of the descent engine is rotating and has to be calculated based on the original setting of the REFSSMMAT, the place we are in the lunar orbit and the orientation of the LM compared to the moon's surface.
The landing of Apollo 11 begins with the crew oriented to look down at the moon's surface with the descent engine bell pointed in the direction of the orbit. Later in the descent, the LM pitches around so that the astronaut's feet are pointed down at the surface and they are looking forward in the direction they are moving. All these occur while the stable member stays fixed in inertial space.
As you can now see, the direction of acceleration we have to input to the AGC, which are given strictly to the X, Y and Z direction of the stable member, have to be manipulated based on all the factors above.
One simplification is that we don't care how the LM moves from face down to face forward since the stable member doesn't change at all. What does matter, however, is the effect of that movement on the thrust vector relative to the 1/6 G acceleration of the moon on the spacecraft. That must be factored in, both by the AGC in computing the burn and pitch up timing, but also in our injected acceleration.
We will need to issue AGC commands to look at the orientation of the stable member at the time we begin the engine burn, calculate how this rotates as we move around the orbital path/descent path, and apply acceleration appropriate to that.
So much more complicated that simply delivering input on the axis of the descent engine bell of the LM. We expect we will even see an alarm thrown after we think we have landed, unless we provide acceleration at 1/6 G in the proper direction to match where the moon surface is relative to the stable member.
The original version of the block II computer had a key on the DSKY marked STBY that was used to put the computer into sleep mode (and bring it out again). Soon, it was replaced by the label PRO and was used for two purposes - standby same as before, and signaling PROceed when the astronaut was ready to approve some action proposed by the computer.
Prior to having a PRO key, the astronaut had to push VERB, push 3, push 3, and push ENTR to confirm a proposed action. When there are a sequence of requests from the program, such as during the start of a landing sequence, having to mash four keys to give assent took time. In some cases, the astronaut had just five seconds to approve the start of a burn; if the VERB 33 ENTR had a typo, it might be impossible to redo it fast enough.
The original STBY function is purely done in hardware. Once the software sets a bit that allows standby, the key itself was wired directly to circuitry that dropped the switchable 14V and 4V supplies, leaving only a base level of power for functions such as pulse generation needed by other hardware in the spacecraft.
The new second use of the key for approving action requires that the software be able to see when the key is pressed. This was accomplished by a small hardware change that routed the key signal from the standby circuitry to set a previously unused bit in one of the I/O channels. Our AGC, the last of the prototypes before the flight versions, did not have the PRO key wired to the channel yet.
We recreated the change, using wire wrap, such that when my DSKY has its PRO key pushed, it sets the bit in the channel to approve a program proposed action, but will still switch off the computer if held long enough to activate the standby circuit.
Challenges running the PDI program for the Apollo 11 landing
We have been able to run program 63, the way you accomplish Powered Descent Initiation (PDI) to lower the orbit of the LM and transition through to landing on the surface. However, once the program sets up for the burn of the descent engine and is given permission by the PRO key to fire, the lack of acceleration is detected. The software flashes Verb 97 which means thrust failure and we can't slow down or reach the surface.
Marc set up an accelerometer substitute using an ATMEL processor and three potentiometers, allowing us to indicate (crudely) that acceleration is occurring on any or all of the three axes. However, the question of which direction the acceleration should be injected is not a simple one. In fact, the answer necessary to bring the LM to the ground with zero forward velocity is devilishly hard to calculate.
The inertial measurement unit of the LM (and CM) consists of a platform (called the stable member) which has three gyroscopes and three accelerometers (PIPAs) mounted so that X, Y and Z rotation and acceleration can be measured.
This stable member is suspended on three concentric gimbals so that regardless of the rotation or movement of the spacecraft, the stable member will remain locked in the same position relative to the milky way galaxy. It drifts slightly over time, which is why it is aligned using the star sighting telescope to return it to its ideal position.
Each gimbal is used to read off rotation of the spacecraft around the stable member in one axis. For technical reasons, having only three gimbals means that the inner and outer gimbals can move so they are parallel to each other. This is bad because a rotation can't be assigned to only one axis, it instead is seen as a movement on both axes simultaneously. This foils the ability to recognize movement properly and is called Gimbal Lock.
Certain rotations of the spacecraft can move it so that it approaches or reaches Gimbal Lock. Once in gimbal lock, the stable member must be reinitialized and star sightings are necessary to align it, a time consuming process. To avoid this, the people in Flight Planning (and in Mission Control) have to plan out the orientation of the stable member so that all the planned maneuvers of the spacecraft can be made without approaching gimbal lock.
Consider the way the spacecraft evens out heat and cold as it flies between earth and moon - the Passive Thermal Control mode - where the spacecraft has to spin on an axis like food on a barbecue spit, with the axis at right angles to the sun. Turning one rotation every two minutes means the stable member is rotating on its gimbals at the same rate and that motion can't be allowed to cause gimbal lock.
At many times during the mission, the orientation of the stable member relative to the galactic background is changed to put it in the best position for the maneuvers ahead. The orientation information is sent up as a REFSSMMAT, a reference stable member matrix either vocally or by the ground using 'remote control' to press DSKY keys to enter the new data in the AGC.
The flight director/attitude indicator display (the 8 ball) is tied to the stable member, thus it is showing where the stable member is sitting while the spacecraft rotates around it. That works out find when in a straight line trajectory such as during translunar coasting, but introduces problems when in orbit around the earth or the moon.
Since the stable member remains fixed relative to the background of the universe, the orbital path will cause the member to appear to rotate as the spacecraft circles the globe. If we start with the stable member parallel to the moon's surface at one point and leave the spacecraft in that same position, as we orbit our nose will appear to pitch up then nose over until we seem to be upside down relative to the moon when we are at the opposite point of the orbit, gradually rotating again until we are flat and parallel at the starting point of the orbit.
The flight director/attitude indicator will rotate around with the stable member (and spacecraft). If we cause the spacecraft to pitch at exactly the right rate, then the spacecraft 'floor' will always be pointing down towards the moon. Our stable member, as a consequence, will be rotating as we orbit and our 8-ball won't make sense.
A special device named ORDEAL will permute the 8-ball so that it seems to match the 'floor' of our spacecraft, and cause the correct pitch rate, so that the pilot does not see the stable member rotating. However, regardless of the tweaked display, the reality of an orbit means that the stable member is rotating while we circle the moon.
That means that the accelerometers idea of X, Y and Z is based on the stable member and are rotating compared to our spacecraft. During the landing, therefore, the direction of the thrust vector of the descent engine is rotating and has to be calculated based on the original setting of the REFSSMMAT, the place we are in the lunar orbit and the orientation of the LM compared to the moon's surface.
The landing of Apollo 11 begins with the crew oriented to look down at the moon's surface with the descent engine bell pointed in the direction of the orbit. Later in the descent, the LM pitches around so that the astronaut's feet are pointed down at the surface and they are looking forward in the direction they are moving. All these occur while the stable member stays fixed in inertial space.
As you can now see, the direction of acceleration we have to input to the AGC, which are given strictly to the X, Y and Z direction of the stable member, have to be manipulated based on all the factors above.
One simplification is that we don't care how the LM moves from face down to face forward since the stable member doesn't change at all. What does matter, however, is the effect of that movement on the thrust vector relative to the 1/6 G acceleration of the moon on the spacecraft. That must be factored in, both by the AGC in computing the burn and pitch up timing, but also in our injected acceleration.
We will need to issue AGC commands to look at the orientation of the stable member at the time we begin the engine burn, calculate how this rotates as we move around the orbital path/descent path, and apply acceleration appropriate to that.
So much more complicated that simply delivering input on the axis of the descent engine bell of the LM. We expect we will even see an alarm thrown after we think we have landed, unless we provide acceleration at 1/6 G in the proper direction to match where the moon surface is relative to the stable member.
Wednesday, June 19, 2019
Interacting with the running Apollo Guidance Computer using DSKY
Testing the DSKY substitute
We hooked up the DSKY substitute to the Apollo Guidance Computer and verified that all the lights, displays and keys worked properly. We now have a physical manifestation of the interface used by the astronauts to interact with the computer.
Video of DSKY working with AGC
Testing Ben Krasnow's electroluminescent panel
Ben (Applied Science YouTube channel) came over with the EL panel he built to NASA specs and we worked to interface it to my DSKY substitute. After a bit of work, I got it mirroring the display on my DSKY as the AGC ran various Apollo era software.
Video of Applied Science electroluminescent panel used with AGC
It was great to see a real electroluminescent display providing the output from the computer. Realism of the interaction took a step up.
Using the DSKY with the AGC
We ran several portions of the Apollo 11 LM mission, using the DSKY to interact with the Luminary 99 flight software that flew in the spacecraft. This included calculating the rendezvous burn after ascent from the lunar surface and of course the powered descent down to the surface. No 1202 alarms for us, because we didn't have a simulated Rendezvous Radar feeding pulses to overload the computer.
We hooked up the DSKY substitute to the Apollo Guidance Computer and verified that all the lights, displays and keys worked properly. We now have a physical manifestation of the interface used by the astronauts to interact with the computer.
Video of DSKY working with AGC
Testing Ben Krasnow's electroluminescent panel
Ben (Applied Science YouTube channel) came over with the EL panel he built to NASA specs and we worked to interface it to my DSKY substitute. After a bit of work, I got it mirroring the display on my DSKY as the AGC ran various Apollo era software.
Video of Applied Science electroluminescent panel used with AGC
It was great to see a real electroluminescent display providing the output from the computer. Realism of the interaction took a step up.
Using the DSKY with the AGC
We ran several portions of the Apollo 11 LM mission, using the DSKY to interact with the Luminary 99 flight software that flew in the spacecraft. This included calculating the rendezvous burn after ascent from the lunar surface and of course the powered descent down to the surface. No 1202 alarms for us, because we didn't have a simulated Rendezvous Radar feeding pulses to overload the computer.
Monday, June 17, 2019
Erasable (core memory RAM) of Apollo Guidance Computer repaired and working properly
New flaw in Current Switch module
After having replaced the two failed diodes in this module, we began testing it with pulses to ensure that it worked properly at the currents and timing needed to drive core memory. This module consists of a number of large ferrite cores with four sets of windings on each. One winding is tied to all the cores and is used to switch them all back to the 'off' magnetic orientation. A second winding is used to select a core by switching it to the 'on' orientation.
Certain cores are selected by flipping them on. This flip of magnetic state induces a pulse in the third wiring, which causes a transistor to conduct to drive current through an X or Y addressing line in the B12 core memory module. Later, when all the cores are reset by the first winding, only the ones that had been selected will flip back. This induces a pulse in the fourth winding, turning on a transistor to drive current in the opposite direction through the X or Y addressing line.
This scheme is clever because the core retains the selected address from when it is selected. This selection induced the pulse to read out a word of the B12 module but also remembers which address lines were selected. The AGC logic does not have to retain the address in a register until later in the memory cycle when the word of erasable memory is rewritten - the selection cores themselves hold that information.
One one of the cores, we had a 4 ohm short in some module I/O pins that shouldn't be connected at all. After excavating the potting material around the suspicious area, we quickly excluded simple to repair causes such as shorting wires, shorted interconnect, etc. We found that the first and fourth windings were somehow shorting inside the core - these are the reset winding and the winding that produces a pulse when a previously selected core is reset.
Careful examination showed us that it would be impossible to extract the core/windings from inside the cordwood module where they were epoxied in place. At least, impossible to remove it without severe damage to that core and its windings.
We have a suitable replacement core but putting four windings of 50, 32, 32 and 20 turns would be extremely challenging given the small diameter of the core. Fortunately, Marc and Mike figured out a clever hack to give us an equivalent functionality. It began by completely disconnecting the fourth winding, the one that switches on a transistor when a selected core is reset. That effectively cured the short since the shorted winding was no longer connected to anything.
Then, we installed a transistor of opposite polarity tied to the third winding, the one that is normally used only while the core is selected. When the selection pulse flips the core, the third winding sees a positive direction pulse, causing the attached NPN transistor to conduct. When the core is later reset, the pulse is in the negative direction thus the transistor doesn't turn on.
Yet, if we hook that third winding to a PNP transistor as well as to its NPN transistor, the pulses from the windings will cause one transistor to turn on with a positive going pulse, and the other transistor to turn on with the negative going pulse. This produces the same behavior as the original circuit, but requires only three windings instead of four. That is fortunate because we don't have four useful windings.
We inserted a small PNP transistor and did some rewiring, which gave us a module that passed all tests with flying colors. We were then ready to install the erasable memory driver modules, this current switch module, and our erasable memory (core stack) into the AGC.
Archiving prior contents of the B12 erasable memory
We still have the flaw in the core memory stack - the inhibit wire for one of the data bits (bit 16) is an open circuit. In the read portion of a memory cycle, the selected X and Y address lines flip all cores of that word to zero. Any of the bits in that word which had a 1 stored in it will flip, causing a pulse to be detected by the sense amplifiers. This is how core memory does a read, by destructively flipping the word to zero.
During the reset pulse from the current switch module core circuits, the cores for the selected X and Y address lines, those whose current switch cores had been set on, will flip all the bits of the word to a 1 state. However, we don't want them to be 1, as some should be 0. That is the purpose of the inhibit wire - a signal on the inhibit wire will block the reset from flipping that particular bit to 1.
Reading data involves erasing it first, then rewriting the original value (or putting a new value in if this is a write operation). With a bad inhibit wire, bit 16 will be rewritten (or written) to 1 regardless of our desired value. Thus the loss of the inhibit wire renders that bit useless throughout all 2K words of erasable memory.
If the inhibit wire damaged happened after the computer was last powered down, then it hadn't yet forced a 1 into every bit 16 in the core stack. We had the opportunity to read the contents of the core memory correctly, since the read portion of a memory access doesn't use the inhibit wire. However, this is a one shot opportunity, as the process of reading any word will jam in the 1 in bit 16 during rewrite.
We powered up with memory access blocked through the test connector, put the machine in single instruction mode, and tested by reading a few memory addresses. We did find that bit 16 sometimes had the value 0, sometimes 1, verifying that the damage to the module was not present when it was being accessed 40-50 years ago.
We wrote down the values we saw in the test locations, then used Mike's test monitor to run through all of memory and retrieve the values. Folding in the few manual reads gave us a complete file of the prior contents of our memory module B12.
Mike studied the contents and could confirm that this was running a version of Aurora software, similar but not identical to the release for which there is an archived image. He could decode the contents of the DSKY display. It showed that the last command entered was a coarse alignment of the IMU (gyroscopes and accelerometers) to angles of 0, 0, and 90. The Gimbal Lock warning light was on, which may have been the event that caused the operator to perform a coarse alignment. Finally, the code stores a latitude when aligning the gyros - it was the location of the Johnson Spacecraft Center in Houston.
Rewiring to swap parity and data bits in B12 core memory module
Now that the prior contents are safely archived, we can implement our modification to make the core erasable memory functional again. Our great fortune is that the erasable memory module contains an error detection method called parity, to capture the cases where some radiation event might have flipped a data bit randomly.
This works by adding a sixteenth bit to each word - data bits are 0 to 14 plus 16, with bit 15 representing parity. The rule for parity is that the number of bits in a word which are 1 has to be an odd quantity. If not, the parity bit is set to 1 thus making the entire word have an odd count. if the data bits themselves have an odd count of 1s, the parity bit is set to 0.
At the end of a read from a word of core memory, the processor counts the 1 bits, determines if parity should have been 1 or 0 and then compares it to the parity bit read from memory. Mismatches raise an erasable memory parity alarm.
The erasable memory is quite reliable, especially since our AGC is not out in space subject to radiation events. We therefore don't really need parity checking. That gives us a working bit 15 which we can substitute for the broken data bit 16. As long as we can disable the parity checking, blocking the alarm, we will have a working memory.
Some backplane wiring was introduced to swap bits 15 and 16, as well as block the signal that performs a parity check. With the wirewrap changes completed, we closed up the AGC and tested. Success! The erasable memory is fully functional and we can run software fully out of the repaired erasable memory and the core rope simulator boxes providing the fixed (core rope) read only memory, just as the machine
After having replaced the two failed diodes in this module, we began testing it with pulses to ensure that it worked properly at the currents and timing needed to drive core memory. This module consists of a number of large ferrite cores with four sets of windings on each. One winding is tied to all the cores and is used to switch them all back to the 'off' magnetic orientation. A second winding is used to select a core by switching it to the 'on' orientation.
Certain cores are selected by flipping them on. This flip of magnetic state induces a pulse in the third wiring, which causes a transistor to conduct to drive current through an X or Y addressing line in the B12 core memory module. Later, when all the cores are reset by the first winding, only the ones that had been selected will flip back. This induces a pulse in the fourth winding, turning on a transistor to drive current in the opposite direction through the X or Y addressing line.
This scheme is clever because the core retains the selected address from when it is selected. This selection induced the pulse to read out a word of the B12 module but also remembers which address lines were selected. The AGC logic does not have to retain the address in a register until later in the memory cycle when the word of erasable memory is rewritten - the selection cores themselves hold that information.
One one of the cores, we had a 4 ohm short in some module I/O pins that shouldn't be connected at all. After excavating the potting material around the suspicious area, we quickly excluded simple to repair causes such as shorting wires, shorted interconnect, etc. We found that the first and fourth windings were somehow shorting inside the core - these are the reset winding and the winding that produces a pulse when a previously selected core is reset.
 |
| Shorted windings in B11 Current Switch Module circuit |
We have a suitable replacement core but putting four windings of 50, 32, 32 and 20 turns would be extremely challenging given the small diameter of the core. Fortunately, Marc and Mike figured out a clever hack to give us an equivalent functionality. It began by completely disconnecting the fourth winding, the one that switches on a transistor when a selected core is reset. That effectively cured the short since the shorted winding was no longer connected to anything.
Then, we installed a transistor of opposite polarity tied to the third winding, the one that is normally used only while the core is selected. When the selection pulse flips the core, the third winding sees a positive direction pulse, causing the attached NPN transistor to conduct. When the core is later reset, the pulse is in the negative direction thus the transistor doesn't turn on.
Yet, if we hook that third winding to a PNP transistor as well as to its NPN transistor, the pulses from the windings will cause one transistor to turn on with a positive going pulse, and the other transistor to turn on with the negative going pulse. This produces the same behavior as the original circuit, but requires only three windings instead of four. That is fortunate because we don't have four useful windings.
We inserted a small PNP transistor and did some rewiring, which gave us a module that passed all tests with flying colors. We were then ready to install the erasable memory driver modules, this current switch module, and our erasable memory (core stack) into the AGC.
 |
| PNP transistor wired to third winding of circuit |
We still have the flaw in the core memory stack - the inhibit wire for one of the data bits (bit 16) is an open circuit. In the read portion of a memory cycle, the selected X and Y address lines flip all cores of that word to zero. Any of the bits in that word which had a 1 stored in it will flip, causing a pulse to be detected by the sense amplifiers. This is how core memory does a read, by destructively flipping the word to zero.
During the reset pulse from the current switch module core circuits, the cores for the selected X and Y address lines, those whose current switch cores had been set on, will flip all the bits of the word to a 1 state. However, we don't want them to be 1, as some should be 0. That is the purpose of the inhibit wire - a signal on the inhibit wire will block the reset from flipping that particular bit to 1.
Reading data involves erasing it first, then rewriting the original value (or putting a new value in if this is a write operation). With a bad inhibit wire, bit 16 will be rewritten (or written) to 1 regardless of our desired value. Thus the loss of the inhibit wire renders that bit useless throughout all 2K words of erasable memory.
If the inhibit wire damaged happened after the computer was last powered down, then it hadn't yet forced a 1 into every bit 16 in the core stack. We had the opportunity to read the contents of the core memory correctly, since the read portion of a memory access doesn't use the inhibit wire. However, this is a one shot opportunity, as the process of reading any word will jam in the 1 in bit 16 during rewrite.
We powered up with memory access blocked through the test connector, put the machine in single instruction mode, and tested by reading a few memory addresses. We did find that bit 16 sometimes had the value 0, sometimes 1, verifying that the damage to the module was not present when it was being accessed 40-50 years ago.
We wrote down the values we saw in the test locations, then used Mike's test monitor to run through all of memory and retrieve the values. Folding in the few manual reads gave us a complete file of the prior contents of our memory module B12.
Mike studied the contents and could confirm that this was running a version of Aurora software, similar but not identical to the release for which there is an archived image. He could decode the contents of the DSKY display. It showed that the last command entered was a coarse alignment of the IMU (gyroscopes and accelerometers) to angles of 0, 0, and 90. The Gimbal Lock warning light was on, which may have been the event that caused the operator to perform a coarse alignment. Finally, the code stores a latitude when aligning the gyros - it was the location of the Johnson Spacecraft Center in Houston.
Rewiring to swap parity and data bits in B12 core memory module
Now that the prior contents are safely archived, we can implement our modification to make the core erasable memory functional again. Our great fortune is that the erasable memory module contains an error detection method called parity, to capture the cases where some radiation event might have flipped a data bit randomly.
This works by adding a sixteenth bit to each word - data bits are 0 to 14 plus 16, with bit 15 representing parity. The rule for parity is that the number of bits in a word which are 1 has to be an odd quantity. If not, the parity bit is set to 1 thus making the entire word have an odd count. if the data bits themselves have an odd count of 1s, the parity bit is set to 0.
At the end of a read from a word of core memory, the processor counts the 1 bits, determines if parity should have been 1 or 0 and then compares it to the parity bit read from memory. Mismatches raise an erasable memory parity alarm.
The erasable memory is quite reliable, especially since our AGC is not out in space subject to radiation events. We therefore don't really need parity checking. That gives us a working bit 15 which we can substitute for the broken data bit 16. As long as we can disable the parity checking, blocking the alarm, we will have a working memory.
Some backplane wiring was introduced to swap bits 15 and 16, as well as block the signal that performs a parity check. With the wirewrap changes completed, we closed up the AGC and tested. Success! The erasable memory is fully functional and we can run software fully out of the repaired erasable memory and the core rope simulator boxes providing the fixed (core rope) read only memory, just as the machine
Main spacecraft connector and patch panel finished, accelerometer simulator plugged in
Main connector
All connections between the spacecraft and the Apollo Guidance Computer take place through the main connector (A51). This has 360 pins, six rows high and 60 columns wide. The DSKY itself connects through A51, which made this a priority to implement.
As I detailed in an earlier post, this is a labor intensive cleaning process but finally we had all of them ready for a male connector to be inserted. We had designed an aluminum plate to hold the 360 Malco mini-wasp pins and to position a printed circuit board we designed atop it. The pins were soldered to the PCB and the far end held eight HD50 connectors (familiar to those who used these as SCSI cables).
The connector was inserted with a partial population of pins - those we needed for our demonstrations. We will finish installing the remaining pins later. The eight cables were run from the PCB out to the patch panels that gave us the same six row by 60 column array as banana plug receptacles. We used this to hook up the DSKY and simulated switches and other spacecraft systems.
Accelerometer simulator developed
Marc designed an Arduino based module that would simulate the accelerometers of the LM spacecraft. These Pendulous Integrating Pulse Accelerometers (PIPAs) swing a small pendulum back and forth kicked by three pulses in one direction followed by three in the other. When no external acceleration exists, the PIPA outputs a steady stream of three positive then three negative pulses.
The AGC software checks for this pattern and throws up a Program alarm (212) if they are not present, since that means the inertial navigation and digital autopilot functions can't be performed. Marc's device would deliver this pulse stream into the appropriate pins of the A51 connector, through our patch panel. He also designed it to allow him to add extra pulses in either direction to reflect acceleration of the LM.
All connections between the spacecraft and the Apollo Guidance Computer take place through the main connector (A51). This has 360 pins, six rows high and 60 columns wide. The DSKY itself connects through A51, which made this a priority to implement.
As I detailed in an earlier post, this is a labor intensive cleaning process but finally we had all of them ready for a male connector to be inserted. We had designed an aluminum plate to hold the 360 Malco mini-wasp pins and to position a printed circuit board we designed atop it. The pins were soldered to the PCB and the far end held eight HD50 connectors (familiar to those who used these as SCSI cables).
 |
| Connector attached to AGC carrying 360 signals out for our access |
Accelerometer simulator developed
Marc designed an Arduino based module that would simulate the accelerometers of the LM spacecraft. These Pendulous Integrating Pulse Accelerometers (PIPAs) swing a small pendulum back and forth kicked by three pulses in one direction followed by three in the other. When no external acceleration exists, the PIPA outputs a steady stream of three positive then three negative pulses.
 |
| PIPA simulator plugged into patch panel |
Retrieving previously lost Apollo programs from a museum's core rope modules
Archiving
We visited the Computer History Museum which has a pair of core rope modules containing the Retread 50 software. These are part of their exhibition of an Apollo Guidance Computer, but we arranged to archive the contents of the modules using our working AGC, in support of their software history effort and to deepen understanding of what is in their artifacts.
Before we connected these to our AGC, it was important to verify that there were no open circuits on the strand and module select lines or other inputs as our computer could be damaged if that was the case. Following a long test plan, we determined that the rope modules were safe to insert.
We first started the AGC in single instruction mode, turned on the core rope access and tried a couple of accesses to see if they appeared to be delivering valid contents, based on the known archive of the predecessor Retread 44 software. It matched.
Mike then used his test monitor tool to read through all of the core rope modules and save their contents. We did spot some parity errors which were confine to specific parts of the modules. In fact, all errors were in the first (B1) module and they were in strands 1 and 9. Modules have 12 strands each holding 512 words of data (thus 6K capacity per module).
For strand 1, a particular data bit was always reading as a 1 value. This appears to be from a failed component inside the core rope module. We did not open the modules as they are artifacts that should not be disturbed. When a word legitimately had a 1 value in that bit, no parity error ensued, but every location that intended a 0 value would trigger a parity alarm.
Strand 9 had more erratic results, but again for just a single bit location in every word. That meant it too could be recovered by looking at the parity bit value.
We wrote a quick post-processing routine to use the parity bit to interplate the correct value of our bad bit. In early programs such as Retread, they left any unused words unwired - thus they would legitimately produce a parity error with all zero results. We had to find those unused words since we had improperly corrected it to have a non-zero word value.
With the data recovered, it could be shared and analyzed. Many of the expected changes were there - since Retread 44 was written on the emulator before the first block II hardware was built, it had some code that failed on real hardware due to inaccuracies in the emulator. All of these flaws in code were corrected, many as were expected but some through alternate changes.
There is some added code, parts that are seen in later (Aurora) software but some that are unique to this previously lost software version. Analysis continues
We hope to archive other lost software versions as owners of other rope modules permit us to plug them in to our system.
We visited the Computer History Museum which has a pair of core rope modules containing the Retread 50 software. These are part of their exhibition of an Apollo Guidance Computer, but we arranged to archive the contents of the modules using our working AGC, in support of their software history effort and to deepen understanding of what is in their artifacts.
Before we connected these to our AGC, it was important to verify that there were no open circuits on the strand and module select lines or other inputs as our computer could be damaged if that was the case. Following a long test plan, we determined that the rope modules were safe to insert.
We first started the AGC in single instruction mode, turned on the core rope access and tried a couple of accesses to see if they appeared to be delivering valid contents, based on the known archive of the predecessor Retread 44 software. It matched.
Mike then used his test monitor tool to read through all of the core rope modules and save their contents. We did spot some parity errors which were confine to specific parts of the modules. In fact, all errors were in the first (B1) module and they were in strands 1 and 9. Modules have 12 strands each holding 512 words of data (thus 6K capacity per module).
For strand 1, a particular data bit was always reading as a 1 value. This appears to be from a failed component inside the core rope module. We did not open the modules as they are artifacts that should not be disturbed. When a word legitimately had a 1 value in that bit, no parity error ensued, but every location that intended a 0 value would trigger a parity alarm.
Strand 9 had more erratic results, but again for just a single bit location in every word. That meant it too could be recovered by looking at the parity bit value.
We wrote a quick post-processing routine to use the parity bit to interplate the correct value of our bad bit. In early programs such as Retread, they left any unused words unwired - thus they would legitimately produce a parity error with all zero results. We had to find those unused words since we had improperly corrected it to have a non-zero word value.
With the data recovered, it could be shared and analyzed. Many of the expected changes were there - since Retread 44 was written on the emulator before the first block II hardware was built, it had some code that failed on real hardware due to inaccuracies in the emulator. All of these flaws in code were corrected, many as were expected but some through alternate changes.
There is some added code, parts that are seen in later (Aurora) software but some that are unique to this previously lost software version. Analysis continues
We hope to archive other lost software versions as owners of other rope modules permit us to plug them in to our system.
Sunday, June 9, 2019
Finishing equipment to read core rope programs and drive the AGC with real Apollo software
Core Rope Simulator box debugging
The core rope simulator boxes that came with the Apollo Guidance Computer were built by Raytheon and designed to cable up to a large console that would host the ROM program in rewritable (RAM) storage. We only had the two boxes inserted in place of the AGC core rope modules. not the large console nor the cables.
After Ken had reverse engineered the boxes and developed a Beaglebone-based driver to serve up the ROM programs to the boxes, we set into the process of debugging. First up, we had to find and repair about a dozen faults, many of them broken wires, as well as reshape IC leads to work in the very unreliable DipStik connectors used in the box.
After the Raytheon hardware was working it was time to test the function of Ken's driver box. It mostly worked right out of the chute, but we had to chase down some defects that caused errors reading certain blocks of memory.
The core rope modules that are normally used with the AGC consist of six separate modules which represent six blocks of 6K words. We will call them R1, R2, S1, S2, T1 and T2. The lowest addresses are in R1 and the highest contained in T2. The faults we experienced were when accessing S2, T1 and T2.
The Raytheon box would latch in the request for a particular module, then which of the 512 cores in that module should be read and which of 12 words from that module are passed into the computer sense amplifiers. What we found was that bad addresses were being latched due to transient short pulses that were due to poor logic design, a race hazard.
We could see the glitch, a pulse on the order of 100 ns compared to the real pulses that occur later and are microseconds in duration. The particular way that the addresses were decoded inside the Raytheon boxes allowed these to pass to the trigger that set flipflops, even though some of the address information had not yet made it through the other gates.
The solution was to add a small capacitor to the trigger line, such that it would ignore pulses that were too short but still lock in addresses when the desired signals arrived. The time constant that would block the glitch kept the integrated very short signal from climbing high enough to trigger the flipflop.
When we inserted a capacitor that was about 100 times too large, it began to delay the real signals - as their leading edge was spread out over a long slope. This caused some other addressing issues, but a capacitor that was just a couple times larger than the glitch did the job perfectly.
These boxes were serial number 3, but it is hard to see how they could have worked correctly on an Apollo Guidance Computer. That is, when the ROM was more than half full. They might have sufficed for small programs that remained in sections R1, R2 and S1. In any case, the core rope simulator works just fine now, letting us run real software such as Luminary 99 that was installed in the LM for the Apollo 11 mission.
Core Rope Module jumpers designed and manufactured
We intend to read existing core rope modules from both museums and private collectors, but a few early sets don't use all six rope modules. The code can fit in just two for Retread 50, as an example, thus four slots are empty. To work properly, jumper plugs must be installed in the four unused slots.
We didn't have any jumpers, but we could work out what was needed. Mike designed some connector housings, built them using a 3D printer, and populated them with the Samtec built Malco Mini-Wasp connectors. He installed small circuit board to complete the jumper circuit, and added handles for insertion and removal. These worked fine, setting us up for the opportunities ahead to read and archive the contents of core rope modules.
The core rope simulator boxes that came with the Apollo Guidance Computer were built by Raytheon and designed to cable up to a large console that would host the ROM program in rewritable (RAM) storage. We only had the two boxes inserted in place of the AGC core rope modules. not the large console nor the cables.
After Ken had reverse engineered the boxes and developed a Beaglebone-based driver to serve up the ROM programs to the boxes, we set into the process of debugging. First up, we had to find and repair about a dozen faults, many of them broken wires, as well as reshape IC leads to work in the very unreliable DipStik connectors used in the box.
After the Raytheon hardware was working it was time to test the function of Ken's driver box. It mostly worked right out of the chute, but we had to chase down some defects that caused errors reading certain blocks of memory.
The core rope modules that are normally used with the AGC consist of six separate modules which represent six blocks of 6K words. We will call them R1, R2, S1, S2, T1 and T2. The lowest addresses are in R1 and the highest contained in T2. The faults we experienced were when accessing S2, T1 and T2.
The Raytheon box would latch in the request for a particular module, then which of the 512 cores in that module should be read and which of 12 words from that module are passed into the computer sense amplifiers. What we found was that bad addresses were being latched due to transient short pulses that were due to poor logic design, a race hazard.
We could see the glitch, a pulse on the order of 100 ns compared to the real pulses that occur later and are microseconds in duration. The particular way that the addresses were decoded inside the Raytheon boxes allowed these to pass to the trigger that set flipflops, even though some of the address information had not yet made it through the other gates.
The solution was to add a small capacitor to the trigger line, such that it would ignore pulses that were too short but still lock in addresses when the desired signals arrived. The time constant that would block the glitch kept the integrated very short signal from climbing high enough to trigger the flipflop.
When we inserted a capacitor that was about 100 times too large, it began to delay the real signals - as their leading edge was spread out over a long slope. This caused some other addressing issues, but a capacitor that was just a couple times larger than the glitch did the job perfectly.
These boxes were serial number 3, but it is hard to see how they could have worked correctly on an Apollo Guidance Computer. That is, when the ROM was more than half full. They might have sufficed for small programs that remained in sections R1, R2 and S1. In any case, the core rope simulator works just fine now, letting us run real software such as Luminary 99 that was installed in the LM for the Apollo 11 mission.
Core Rope Module jumpers designed and manufactured
We intend to read existing core rope modules from both museums and private collectors, but a few early sets don't use all six rope modules. The code can fit in just two for Retread 50, as an example, thus four slots are empty. To work properly, jumper plugs must be installed in the four unused slots.
We didn't have any jumpers, but we could work out what was needed. Mike designed some connector housings, built them using a 3D printer, and populated them with the Samtec built Malco Mini-Wasp connectors. He installed small circuit board to complete the jumper circuit, and added handles for insertion and removal. These worked fine, setting us up for the opportunities ahead to read and archive the contents of core rope modules.
 |
| Rope jumper connector in place |
 |
| Connector removal using handle |
Subscribe to:
Posts (Atom)