Odd error detected with memory, possible issue on the 1130 MRAM board

BIZARRE ISSUE WITH MEMORY REVERTING TO OLD VALUE

I was running the memory diagnostic with parity stops disabled - the parity issue is a separate problem I am troubleshooting - and the loop advancing through memory was never ending. When I looked closely I found that the code was adding 1 to a stored address in a memory word but would seem to skip back to a prior value. 

It began at some point with an address around 0481 and would step up to 048F before returning to 0481. If I stored a higher address in the memory word, for instance 0491, it would advance to 049F and then return to 0491. I hand stepped the code and it appeared to be correctly advancing the address to 04A0 and stored it, but then jumped back to 0491. 

I then manually displayed and loaded the word where the address was stored - this in memory location 02E7 - and saw very puzzling behavior. I would display the value in 02E7 as 0491, then update it to 04A0. The first time I displayed the content of 02E7, I retrieved 04A0 but then when I displayed the same location again, I saw 0491 returned!

Memory is not a push down stack. Writing a value in a location should totally replace the old value. If it reads as the new value once, it should always read back as the new value unless I rewrite it explicitly back to the old. It should have no way to 'remember' the prior value.

WHAT MIGHT CAUSE THIS?

I did some thinking about mechanisms that might lead to this behavior. I stopped when I had a path that could deliver the old value for a read request, not diving any deeper to find a means that it could accomplish the feat, simply looking at a pathway to get that old value back.

A memory cycle in the IBM 1130 is based on the way that core memory works. Reading the content of a word involves destructively setting the word to all zeroes, detecting which bits were previously set to one. Those bits that had stored a one before the destructive read will cause the core memory system to deliver a pulse into the 1130 where the Storage Buffer Register (SBR) has that bit flipped to be one. 

The second half of the memory cycle will write whatever is in the SBR into the cores. It actually tries to flip every bit of the word to one, but inhibits the flip if the corresponding bit in the SBR is a zero. Thus core memory erases the contents, transferring its value into the SBR and then rewrites the word from the SBR. If the 1130 wants to put a new value in a memory location, it first reads the current value into the SBR and then updates the SBR before writing it to the machine. Every memory cycle has a read followed by a write. 

Thus, the first possible pathway is if somehow the SBR is updated after a successful read, putting a different value and then writing that different value back. This would let my first read of 02E7 successfully pull the value 04A0 into the SBR but somehow during the write it is changed to 0491 and written that way. The next time it is read back, 02E7 returns 0491 and not the value we thought we wrote into it. I don't yet have a way that the 0491 value would be retained in the 1130 and forced into the SBR for the write cycle, but this is a way that reversion could occur. 

Another possibility is if the address lines running into the MRAM chip on my board were to change during the cycle, reads might actually be fetching values in two different words. Thus, if the address 02E7 is sometimes seen as 06E7, two different values could be retained and returned. I don't understand how the first read of 02E7, after we updated it to 04A0, would address one word and subsequent reads would address the the other word location, but this is a way I could pull a previous value from a location.

One final possibility is that the chip itself is defective. It might allow the value in 02E7 to slowly revert to the old value even though it held an updated value for a short time. It might misaddress locations in the chip much like my speculation in the prior paragraph so that two different locations are being retrieved at different times. 

I could watch for the first speculation by using the logic analyzer to record the SBR bits, let the machine get into the loop where it reverts, then examine the trace to see if the write cycle is changing the value of the SBR. 

I could watch for the second speculation but only if the address bits change in the 1130 in the Storage Address Register (SAR). That would happen if I trace the 13 address bits of the SAR and record while the machine is in the reversion behavior. However, if the address bits are flakey on my 1130 MRAM board and not from the SAR, then I wouldn't catch this issue with the logic analyzer. 

I will reflow the address bit soldering points on my board to eliminate the chance that the cause might stem from a poor connection. However, to test for errors inside the MRAM chip itself, I might need to manually do sequences of load and display operations on the 1130 while recording the SBR bits. 

Trying to track down the parity stop failure using my 1130 MRAM board while running IBM's core memory diagnostics

RECAP OF THE FAILURE TO BE STUDIED

When running the IBM 3B1 diagnostic - core memory testing with the code residing in low addresses - the machine will stop with a parity error at the same location retrieving from the same address. The program had executed the loop more than a thousand times before the error occurs. The machine is executing a store instruction in indirect mode, where it puts the data in the location stored in the memory word, rather than into the memory word itself. 

The machine has just fetched the address from the memory word but hadn't begun to store the contents of the accumulator register into that address. The data that was fetched into the Storage Buffer Register (SBR) and the two parity bits don't agree under the odd parity rules of the system. Since my card generates the two parity bits on the fly from the retrieved word, this should not happen. 

TRACING STATE LEADING TO THE FAILURE

I set up the scope to trigger on the setting of a parity stop, while monitoring various other signals such as + Storage Write and the actual output of the parity testing logic in the 1130. The goal is to see what seems incorrect or hints at other places to look, with the ultimate aim of seeing the definitive root cause of the failure. 

The eight data bits of each half of the SBR are cascaded through exclusive-OR (XOR) gates so that we know if there is an even number of bits that are set to 1. The parity bit associated with those 8 data bits is used to ensure that the total 1 bits in the 8 data and 1 parity bits come to an odd total. When the machine fails, it is because one of the halfwords and its parity bit have an even count of 1 bits. 

I consistently saw the diagnostic produce an error where the right halfword (bits 8 to 15) had the mismatch with its parity bit P2. When I put the scope leads on P2 and the output of the XOR cascade for that halfword, the mismatch is now with the first halfword and its parity bit P1. Moving the scope leads to that side and rerunning, I found that the error flipped back to the second halfword again.

I doubt this is a quantum phenomenon where an observed condition works differently than when it is not watched. However, something must be caused by high impedance scope leads attached to one of the nets. I wonder what will happen if both sides are observed at the same time. Will the machine run without parity stops? Seriously, I do need to observe more points simultaneously, which means I have to move over to a logic analyzer. 

More work building 1627 plotter controller card for IBM 1130

FINAL COMPONENTS INSTALLED

I soldered down the linear voltage regulator and the remaining logic chip - a 74LS05 open collector hex inverter. I then placed all the resistors and capacitors onto the board. 

The pads across the top are where the spring contacts will be soldered. Those contacts will slide over the pins on the 1130's backplane board where the 5806223 card from IBM would be installed. Of the 48 positions total across the two sockets, 21 are not used. Three pins are used to deliver power and ground the the card, with the remainder carrying signals between the card, the IBM 1130 system and the plotter (IBM 1627, Calcomp 565 or Calcomp 563). 

Constructing the 1627 plotter controller card for the IBM 1130

REDESIGN USING FPGA DAUGHTERBOARD FOR LOGIC

The original design used surface mount chips including multiple timers and combinatorial logic, matching as closely as feasible the circuit design of the IBM 5806223 card which implemented the controller function for the IBM 1627 plotter when connected to an IBM 1130 mainframe. 

INSERTING COMPONENTS ON PRINTED CIRCUIT BOARD

PCBWAY sponsored the printed circuit card for this redesign, which I received recently and wrote about. I soldered on the daughtercard (Digilent CMOD S7 FPGA board) and one of the integrated circuit chips. As the remaining parts arrive and I have time, I will solder up the rest of the components on the PCB. 

The most time consuming part of construction will be soldering down the contacts that will form the Solid Logic Technology (SLT technology of IBM mainframes from early 1960s) connectors. This card plugs into two SLT slots inside the IBM 1130 system. 3D printed covers finish off the connectors on the card, covering the contacts and guiding the mating of the card into the SLT socket. 

Above you can see the current state of the new card on the left and the earlier version on the right with its connectors at the top. Pads on the new card are visible, onto which the metal contacts are soldered to create the connectors. 

Honing the FPGA logic for the 1130 MRAM board and testing results

FINE TUNING THE STATE MACHINE

There are points where I examine the external signals coming from the IBM 1130 in order to start the read or write portion of a memory cycle, as well as whether to bail out if I find that the +Storage Use signal is not asserted. 

These signals are passed through four clocked flipflops in order to handle the possibility of metastable states since the 1130 could change the signal right at the same time as the clock edge of the FPGA since they are not in a synchronized clock domain together. I had been simply examining the output of the last flipflop for the +Storage Read, +Storage Write and +Storage Use signals but if the signals are glitchy, looking at a single 5 nanosecond point in time might give me a false positive. 

To protect against this, I added four more clocked flipflops to each signal chain and then verify that the last four are all set to 1 in order to treat the signal as an actual positive. If there is a temporary glitch to 0, we just wait another few cycles until all four flipflops are filled with 1 in order to proceed. Similarly, the +Storage Use chain must have the last four flipflops at 0 in order to act as if storage use is not requested for this memory cycle. 

The state machine, once started in a read or write portion of a cycle, will finish its activity and then wait until the 1.8 microsecond cycle portion is almost complete before it begins to look for the 1130 signal to proceed into the other half of the memory cycle. If I find that storage use is not requested, even if the state machine began the read cycle, it will wait long enough to have passed almost fully through the entire 3.6 microsecond memory cycle before it starts looking for +Storage Read again. 

When I look at the logic diagrams for the IA cycle, it shows that the Storage Buffer Register (SBR) is cleared for the entire duration of clock step T0 - from 0 to about 450 ns - and then released so that the sense pulses from my board can set the relevant bits. My previous logic was emitting the set pulses prior to 450 ns so that had to be changed.

I also see that the logic will take the results from the SBR and move them to the ACC register during clock step T2 which occurs from about 900 to about 1350 nanoseconds. Thus my results from the sense pulses must be transferred before 1350 ns. The spec for the sense pulses from the IBM core memory is for them to occur as early as 700 ns and as late as 950 ns which puts my real upper bound at 950 ns. 

I adjusted the timing of the FPGA state machine to fit with those constraints. 

RESULT OF TESTING SHOWED THE SAME PARITY STOP ERRORS

The core memory diagnostic fails at the exact same spot every time. It is in a routine that fills memory with a given pattern and at an address that begins with 0x000011111xxxxxxx and not when it first executes the code. It makes it more than 1500 loops before it encounters the error. 

It fails consistently at certain address values while in full speed RUN mode. However, if I step the 1130 with single instructions it does NOT fail. It is a timing issue that is marginal enough that it is dependent on specific values to run into trouble. In most cases the data shows up exactly as expected in the SBR, even when I view the display panel 

The code is storing the pattern into memory words using a STO I ADDRS instruction, a store of the ACC register into the address that is contained in the memory location ADDRS. It first fetches the value in ADDRS, then performs another memory cycle for indirect addressing (IA cycle) which will set up the Storage Address Register with the new location into which the value in the ACC register will be placed during the next memory cycle E1 (execute cycle 1). 

It does allow me to speculate about how the parity stop is being triggered, however I need to set up the scope to try to trap the situation and trace it back to be certain. 

Parity stop occurs when the exclusive OR of the eight data bits in a halfword plus the parity bit associated with that halfword result in an even count of 1 bits, at the start of the T6 or X6 clock step. T6 is in the middle of the write portion of the memory cycle, whereas I believe the fault that causes the stop is happening earlier. 

Since the parity bits are generated by the contents of the MRAM memory but the bits in the SBR are injected by the sense pulses from my board, if the high order bits are somehow not getting set into the SBR due to the timing issue, but the parity is generated based on the retrieved bits, then we could trigger the parity stop. 

What I don't understand is why the high bits of the SBR are almost always stored correctly, and even in the case that failed it has executed the IA step for the instruction more than 1500 times successfully before. Why is the timing not only marginal but failing specifically at this one time predictably. 

Finding remaining issues on my MRAM board

ENHANCED MY FPGA CODE AND RETESTED

I added some logic that aborts read and write operations if it discovers that the +Storage Use signal is not active. This protects against possible race hazards that might start my state machine although the 1130 does not intend to access core memory. Just before it sets bits in the Storage Buffer Register (SBR) for a read and before it writes new data into the MRAM chip, I check +Storage Use and skip over those actions. It does this by waiting long enough for the entire 3.6 microsecond storage cycle to be complete. 

I also added a step in the state machine that turns off the tristate buffer. The changes were simulated and then a configuration file was generated to load into the FPGA which I did when I returned to the workshop. 

Most importantly, since I saw some signs of retriggering two read cycles within one 1130 read cycle, I added delay logic to the state machine so that it let enough time expire after finishing the read or write that it was almost at the time when the complementary cycle (write after read or read after write) would start, thus not allowing the state machine to be tricked into a double cycle. 

Storage Load and Storage Display functions were tested again and behave correctly. The 1130 mode switch was set to Load and Display to manually access storage, which also behaved properly as it did before. 

I then loaded the core memory diagnostic program using my Console Loader enhancement to the 1130 and ran it. Previously it failed with a Parity Check consistently at one point in the code. It still fails the same way. This is only happening with certain instruction sequences, otherwise the machine runs reasonably large blocks of code correctly. 

If it were an issue where the data contents, the Storage Buffer Register bits, were changing midcycle then whatever was written to MRAM would always have correct parity generated on its later retrieval in a read cycle. The SBR should not be changed during the portion of the read cycle where I am emitting the pulses to update the SBR. If a read were to complete but is repeated due to changing SAR bits in between when the data bits begin storing in 85 ns clusters and when the computed parity is stored in the last 85ns cluster, it could cause the symptoms.

Furthermore, looking at the three parity stop cases above, I see that in every case, it is the right halfword that does not agree with its parity bit P2 and it is likely that bit 8 is being spuriously stored in those cases. Thus I may have an error in my printed circuit board where there is a short between pins 8 and 9 on the FPGA (CMOD S7) board connector or between pins C1A11 and C1B11 on the T1 connector socket. I will check this in addition to looking more closely at the code being executed when the parity failures were caught. 

It is possible that there is an issue with some timing related to when the 1130 changes the Storage Address Register and when I begin a read or write, such that the contents are a blend of the old and new address. The fact that bit 8 is always on in this parity stops is suggestive that this is closer to the root cause than mistiming and SAR changes, but I will have to investigate all of it. 

1627 Plotter controller card redesign - printed circuit board arrives from PCBWAY.COM

PCBWAY.COM SPONSORED THE NEW PCB, BOARD ARRIVED TODAY

I received 5 printed circuit boards for my new design of a controller card to insert into an IBM 1130 system in order to connect an IBM 1627 Plotter. PCBWAY.COM had contacted me on this blog and offered to sponsor some printed circuit fabrication for upcoming projects. I thought this project would be a good opportunity to try this out for both them and me. 

IBM makes a card that is plugged into a logic compartment in a specific slot to add support for the 1627 to the configuration of the system. It is built with their Solid Logic Technology (SLT) components as a double width card. SLT cards plug into a socket on a backplane using a connector that has two rows of 12 pins. The double width SLT card is 3 3/8" wide. 

SLT backplanes are in a compartment, with space for up to 65 single width SLT cards plus some cables that use the same 2 x 12 connectors. Slots are labeled with columns A through N and rows numbered 2 to 7The IBM 1130 has two swing out gates each of which holds three compartments. The 1627 controller card from IBM (part number 5806223) fits into gate A, compartment C1, in slots M4 and M5. 

The card implements logic to handle the input-output instructions for the device at Area Code 5, which is what IBM assigned to the 1627 Plotter. Asking the plotter to move the pen is done by issuing the XIO Write instruction, with bits in the transferred word assigned to lift the pen off the paper, lower the pen onto the paper, move the pen left or right or move the drum with the paper down or up. 

The state of the Plotter is interrogated by issuing an XIO Sense Device instruction which returns a word with bits assigned for various conditions. The card will raise an interrupt request when the pen movement is complete, informing the software so that it can issue additional movement requests. 

The card implements the XIO Write and XIO Sense Device functions for Area Code 5, requests the interrupt on level 3 when the operation is complete, and sends signals to the 1627 to command the various pen movements. 

My redesigned card uses an FPGA board (Digilent CMOD S7) to implement the logic functions, with the card providing the interface chips for signals to and from the IBM 1130 and signals to the 1627 Plotter. It also drops the 12V power rail from the 1130 down to the 5V level needed for my chips and the FPGA board. 

The most complicated part of the card is the contacts that form the connectors which plug into the SLT backplane. The backplane has 24 pins for each of the two sockets, while the card slides over the pins and locks into position. The locking is mechanically complicated and very difficult to reproduce, with the pins having a hook at the end that locks over a bent contact shaped to form a notch which holds the hook. 

My previous version of the card had contacts that mate with the SLT backplane pins but don't latch. The card therefore backs out of the backplane sockets. I need to work out a method to hold the card down into the sockets until I want to pull it out. 

SWITCHING TO USE PCBWAY WAS PAIN-FREE

I have been using other fabrication houses in the past but found the change extremely easy to make. I could use the same settings and design approaches with KiCAD that I used with other fab houses. The requirements for what has to be sent to PCBWAY.COM were very flexible and accommodated the settings I already used. 

The website is easy to use (at least for anyone used to working with printed circuit fab houses), well documented, and has lots of supporting information if you have questions. I defined the size of my four layer printed circuit boards, accepted all of the default choices for copper weights, layer spacing and so forth, making only one change so that the order number is NOT printed on my boards. My design files were uploaded and the site gave me prices and other options. 

It was very intuitive to request a standard manufacturing turnaround (4 to 5 days for my design) and select which shipping method I wanted. I picked DHL but there were other options including Fedex and UPS. 

QUICK MANUFACTURING AND SHIPPING

They offer the ability to select faster manufacturing turnaround and even quicker delivery than the 3-7 days using DHL, but I found the standard choice and DHL to be good enough. I placed the order and once I coordinated the sponsorship with PCBWAY.COM they began manufacturing in the evening of Sunday March 15th. 

The production was completed and the box went out with DHL by March 19th. The estimated time for standard production was 4 to 5 days, which this met with a bit of room to spare. 

They don't control the shippers nor customs processing, but the box arrived on Tuesday March 24th. Quite satisfactory for an international shipment.  

OPENING THE BOX

This came in a plain white box, shrink-wrapped and there was no rattling when the box was shaken. I opened the box to find some white foam padding inside.

I took out the padding and came to the receipt plus the circuit boards vacuum packed in a bubble wrap material. A bag of desiccant was sealed inside to absorb any moisture that might seep in. This is how printed circuit boards should be shipped.

There were actually six boards inside. My guess is that they had a panel with some errors in someone else's card and remade it, but the area with my board was good. The other person received five boards from six panels, so I got a bonus board. 

I cut open the bubble wrap and inspected my boards. They look very well made, certainly matching my design and right up to the standard I expect from an excellent fab house. 

As you can see, it is a relatively simple board, although I do use four layers to give me a solid ground plane. I feel confident that PCBWAY.com would handle far more complex boards and deliver the same quality. Thank you, PCBWAY.COM, for the sponsorship of this board for my project. 

WILL ASSEMBLE PARTS ON THE BOARD SOMETIME LATER

This project is a lower priority activity which I will work on at times when I am in waiting mode on higher priority efforts such as having ordered additional parts. However, such pauses are frequent so I do expect to put the board together in a month or two. I did order the FPGA board and other parts already so that I could even install the components on it in short order. It will be the soldering of the SLT socket on the top that will take quite a bit more time and adjustment, so that may not be worked on until the priority bumps up.