Triumphed over toolchain

DISCOVERED REMNANTS OF DEBUG CORE DATA TACKED ONTO XDC FILE

I found that the process of setting up debug wrote data onto the end of the main constraints file (XDC) which conflicted with attempts to set up a new set of ILA cores with the signal moved to the correct clock domain core. 

If I erased those entries, I could run the synthesis, set up the debug and then run through bit file generation without issue. I haven't been to the shop to test it but I expect that I have resolved everything and can get back to testing.

REASSIGNED THE SPI LINK CLOCK TO A BETTER FPGA PIN

The SPI link provides the link externally, generated by the Arduino. Thus this clock is not related to any clock inside the FPGA. The design of the SPI slave module I leveraged handles the handling of the different clock domains, so I don't have to deal with the signals. However, it is treated as a clock by that portion of the logic in the slave module and that means that Vivado wants to use clock buffers and the special clock lines inside the FPGA chip. 

I had been receiving warning messages about the inadequacy of the clock line and buffer connections to this input pin, so I dug into the more intimate details of the chip and its clock resources to fix this. I found that a subset of the input-output pins are capable of good interface to the clock resources - named CCIO pins. 

I found the list of CCIO pins and identified a few of the IO connector pins on the FPGA board which matched those. By shuffling two signals, I had the SPI SCLK signal hooked to a CCIO pin which resolved all the clock inadequacy messages. 

Dopey toolchain design blocks any progress today

VIVADO WON'T FULLY REBUILD DEBUG CORES, DIES ON ERRORS

I took my corrected design, the one that put the signals in the correct clock domains, fired up and tried to debug but noticed that the app_wdf_data signal that had been in the general logic domain was still tied to the debug core for that clock, not moved to the memory clock domain as it should. 

I removed the debug cores and implemented just fine. I then redid the debug setup but when implementing it throws errors about missing debug cores. No matter what I do, it won't properly build debug cores. 

Vivado is notorious for this sort of misbehavior. It remembers things and has no method to force a full clean build. Workarounds require manual deletion of files, dangerous at best. I wasted the entire session I had available today fighting with the tool rather than debugging my logic.

INSTALLED ANOTHER SPRING AND TOOK OUT THE PRINT SHAFT OF THE 1053

The small spring I attempted to connect last time beckoned, while Vivado was busy failing to implement the logic. I was able to conquer the spring using many different tools and angles of approach. 

The print shaft is the main support that the carrier slides across from left to right, and it has a notch to transfer rotation of the shaft to throw the typeball forward when printing a character. The shaft had corrosion or hard crusted junk on it that was rubbing against the rubber sleeves and restricting carrier movement.

While I reserve the option to replace it with another shaft, I did use emery paper which is removing quite a bit of the coating and making the surface smoother. It won't remove every pit but if the carrier can move freely then I can keep the original shaft on the printer. 

Short debugging run highlights design flaws - signals changed in the wrong clock domain; some Selectric work done

ERRATIC RESULTS IS USUALLY DUE TO TIMING ISSUE

Simulation is not a perfect solution to testing for these issues, particularly since my testbench sets the timing of all the signals driving the logic. In the real design, we have an external clock coming from the Arduino SPI link as well as generated clock modules in the FPGA that determine the relative phase and timing of signals. 

That is the value of having oscilloscopes to look for relative timing of signals as well as logic analyzers to show what is actually occurring inside my mechanism. I was able to dash over to the shop for a couple of hours during which I looked for clues that would explain the erratic behavior I am seeing.

DATA TO WRITE INTO RAM IS SET UP UNDER THE GENERAL LOGIC CLOCK

The internal logic analyzer cores placed in my design by Vivado are segregated by the clock domain in which the signals operate. For the signals I am watching, those are the general logic clock at 50MHz and the memory controller RAM clock at 100MHz. There is also the SPI clock domain which I am not currently monitoring as well as an internal 200MHz clock in the memory controller. 

While I was looking at the RAM access signals, which are in the RAM 100MHz clock domain, I noticed that I wasn't seeing the data being input to the memory controller to write into storage. I then spotted that signal in the other ILA, the one under the general logic 50MHz clock. Aha! It is exactly this sort of design flaw that can produce erratic results, where most of the words have the expected value but some do not. 

RESTRUCTURING THIS PART OF MY DESIGN

I will be carefully planning this portion of my logic, to ensure that any data going into or out of RAM is handled under the general clock at the input to the request FIFO or the output of the response FIFO, while all the memory controller inputs and outputs come from the request FIFO output and response FIFO input since those operate under the RAM clock. 

The data being read and written in RAM comes from one of two broad areas - the disk drive modeling logic or the Arduino SPI link logic. Where I was selecting between the sources and destinations is where the flaws exist. 

SELECTRIC WORK ACCOMPLISHED WHILE REBUILDING FPGA BIT FILES

I decided to monitor two more signals in the internal logic analyzers which mandated synthesizing, implementing and generating bit files again. As that takes almost a half an hour, I went over to the Selectric typewriters, the 1053 Console Printers of the two 1130 systems, to make some progress.

I was able to free up a critical area of the machine that controls spacing, backspacing and tab operations. These levers on the rear of the carrier need a lot of lubrication and exercising to free them up from the years old sludgy oil and grease in the machines. I had it moving freely then observed that a key spring was missing, one that shifts a sliding bar involved in tab operations.

I located a suitable spring and installed it. The carrier is still not moving well, which is due to the corrosion on the round shaft that it slides across. That shaft is keyed and also imparts rotation to swing the typeball up to strike the ribbon and paper. I am skeptical that I can clean up this bar adequately to allow good clean movement of the carrier, so much so that I am looking to find a donor Selectric typewriter whose bar I can salvage. 

Space and tab operations operate for one cycle, with the mechanism pushing the activation lever forward at the end of the cycle. It should latch into place, based on a smaller spring activated lever. One of the activation levers was missing the small spring, thus it would not relatch, causing the machine to take multiple tab or space operations. I found and installed an appropriate spring. 

The space operation depends on a mechanism tilting a bar behind the carrier, but there was a spring that had snapped there. I tried to get it back into position but have not yet been successful. In theory it is easy to attach springs. Tools called spring hooks can grab each end, the spring is maneuvered into place and the ends coaxed into their mounting holes or tabs. 

In reality, most locations do NOT have room to bring in a spring hook on one side. The sides that can be handled often are with a tilt that not ideal. This spring for the space mechanism is in a challenging area, thus I will have to keep at it until I can get both ends secured. The original spring had corroded and broken in the middle. 

RAM access working, but slightly erratic operation of the link still needs some debugging

SHORT SESSION SQUEEZED IN USING NEW DIAGNOSTIC TOOLS

I was able to spot an issue, correct it with only a 30 minute turnaround for a new bitfile for the FPGA. I began to see that most of the returned words on the Unload transaction were what I had written previously with the Load transaction. This shows I am indeed writing to and reading from RAM addresses. 

I did see that the results were mostly correct - the value returned should be the word number which is what I wrote, but some came back incorrect. Worse, I would see the checksum and flag coming back from the FPGA while the Arduino still thought it hadn't finished all 321 words of data. Things are getting out of sync. 

Since it will be a day before I can get back to the shop, I will look over the logic and see where I can tighten up the operation of the state machines to keep everything locked properly together. 

The other thing I haven't yet validated is that the address I am sending is going to the correct RAM locations. I need to know that a read or write to a particular cylinder, head and sector will take place against the correct part of the disk cartridge image. 

Debugging tools working well - just need time to work with them

LED ARRAY PANELS

Somehow I didn't expect the lights to alternate directions, but that makes sense with the most direct wiring path on the panel. Looking at the panel, the top left pixel is number 0. Running down the left edge we have pixels 0 to 15, then it turns around and runs upwards one column over, with 16 at the bottom and 31 at the top. Reversing again, the third column runs from 32 at the top to 47 at the bottom. 

It took a moment to determine what number each array position is associated with, but then it was easy to see the condition of the logic with all those pixels active. Very easy when I have the Arduino send each word of a transaction with a human sized pause - the values going up and down the SPI link are shown clearly during the pause. I can also see the transaction type latched as the related LED lights up. 

INTEGRATED LOGIC ANALYZER WORKING AS WELL

It only took a few minutes poking around until I figured out how to link up to the ILA cores and begin watching the signals. The only complication is that finite state machine names are converted to a hex number, thus I need to look these up before attempting to trigger the logic analyzer on a specific state. 

In just five minutes, before I had to head back to my home, I found a defect and have already corrected the logic (VHDL code) in preparation for my next time at the shop.

Invested time in learning the integrated logic analyzer and other debug capability of Vivado toolchain

PROGRAMMERS ATTITUDE

It is common for programmers to spend time writing code to automate activities that they will otherwise have to accomplish more manually and tediously, if those activities will occur multiple times. The up front investment in writing macros, tools or other programs is recouped when the activities are accomplished many times with far less effort than otherwise would be required.

My original approach to debugging was to choose a limited set of signals to bring out to the external I/O pins of the FPGA board, to which I would connect oscilloscopes or logic analyzers to hunt for signs that illustrated one or more logic design flaws. The time it takes to reselect signals and regenerate the FPGA is about half an hour of wall clock time, but may recur many times as I iterate through signal sets before finally seeing the definitive evidence. 

After poking around at the documentation for Xilinx's Vivado toolchain, I came to the conclusion that investing some time now would not only offset all those random walks through sets of signals in half hour sessions, but would be a useful tool for all future FPGA designs. I thus began reading and experimenting to figure out the capabilities.

INTEGRATED LOGIC ANALYZER CORES

Once a design has been synthesized, it is relatively easy to open the netlist of signals and mark all those that you would want to have available for a logic analyzer - casting a wide flung 'net' to grab any signal that potentially be viewed or used as a trigger. Then the debug setup tool will prepare debug cores to capture these signals. It groups signals into their clock domains and produces one core for each clock. In my case, one core for the signals operating at the normal logic frequency of 50MHz and another for the RAM operating clock of 100MHz. 

These are instantiated onto the FPGA alongside my logic and are connected to over the USB cable from Vivado to the FPGA board once I program the board with the new bitfile. A logic analyzer program runs under Vivado and communicates with the ILA core that was produced. 

VIRTUAL INPUT OUTPUT CORES

Another type of core can be produced, where signals are marked that you wish to read or change from the Vivado software while the FPGA logic is running. This VIO core talks to the Vivado software over the USB cable and with it one can vary inputs and read outputs purely by software. This is interesting because it could be used in lieu of the working disk drive and 1130 computer to produce the control signals and therefore drive my logic through its paces. 

AXI DRIVING CORES

ARM developed Advanced eXtensible Interface - AXI - to allow a more standardized way for modular logic components to interact with others and with processor cores. Many bits of available functionality and intellectual property for use with FPGAs exists with an AXI interface option. Because the interface is so well described, a new design that uses AXI can be debugged by driving it with an AXI core talking to Vivado that will handle all the interface details. I don't expect to need AXI cores, but I discussed it here to round out the list of capabilities built into Vivado.

STEPS TO INTERACT WITH MY SIGNALS

It didn't take long to open my synthesized design, mark the signals I wanted routed to the ILA core, then set up the cores themselves to be included on my board. From that point forward, I just implemented, generated a bit file and created the memory load to put that file into the flash memory on the board in the same way I worked without the debug cores. 

Select netlist signals for use with ILA core

Signals are 'probes' in the ILA

When I connect to the FPGA board to download the bit file to the flash memory, from whence it is loaded whenever the FPGA board is powered up or reset, it also shows me any debug cores that are implemented. Selecting one or more will bring up the software that interacts with ILA or VIO cores. The ILA software is very similar to the logic analyzer capabilities in HP and Tektronix physical analyzers, thus easy to figure out and work. The resulting display is exactly like the waveforms displayed by the simulator built into Vivado, which I have quite a bit of experience using. 

Implemented LED panels plus added diagnostic output pins to debug FPGA

512 LED ARRAYS, TWO AT 16x16, SET UP TO OUTPUT INTERNAL SIGNALS

Due to the error in the design of the leveraged logic for driving the panels, I am only using 510 of the 512 pixels. I hooked up just about every signal in the design that might make sense to observe statically or at slow speed and still had plenty of capacity. I chose to set up the second panel of 256 with a light for each of the 203 cylinder locations, plus lights for the head selected and which of the four sectors is active. 

ADDITIONAL OUTPUT PINS ON PMOD CONNECTORS PUT INTO SERVICE

I knew that some of the pins on the PMOD connectors of the ARTY FPGA board were shared with the other input-output connectors I was already using, but upon close inspection I found that 16 signals, those on the A and B PMODS, were not shared and thus available for additional diagnostic outputs.

These are appropriate for timing or short activation detections where the LEDs are not suitable. I brought out the logic clock as well which will allow me to hook up a logic analyzer if I need to see the interplay of many signals at the same time. 

FUTURE POSSIBILITIES WITH LEARNING CURVE BUT MORE POWER

The Vivado toolchain includes some powerful tools to let me inspect and even inject signals without having to pre-select external connectors for this purpose. These are the Integrated Logic Analyzer (ILA) and Virtual IO (VIO) functionality that I can add into my design. ILA is what it sounds like, a logic analyzer with rich trigger conditions that I could use to follow signals from my PC over the USB cable to the board. VIO lets me see virtual pins and inject signals to drive my design. 

It takes hardware resources to implement these, the more signals you set up for potential connection, the more of the board is consumed. I don't have any idea whether my entire Virtual 2315 logic could coexist with ILA or VIO or both on the Spartan 7 version on my board. The more powerful factor holding me back is the time it takes to learn how to use each of these, time taken away from the project if I could otherwise finish debugging without these tools. 

Having to debug and fix the Neopixel logic I leveraged to drive the panels from the FPGA

USING SOMEONE'S DONATED LOGIC TO DRIVE THE NEOPIXEL ARRAYS

This logic was developed by Blaž Rolih and shared under the MIT License on github. It seemed like a great fit thus I made use of it to add the two 16x16 pixel arrays for diagnostic displays.

# fpga-neopixel

FPGA module for NeoPixel led-strip written in VHDL. Works with ws2812b (RGB) and sk6812 (RGBW).

Made and tested in Xilinx Vivado, with Nexys A7 Artix-7 CSG324 FPGA board with 100 MHz clock but should work on other boards with minor adjustments as well.

My logic is operating at 50MHz and the first issue I discovered and had to fix was that its customization for different clock speeds wasn't complete. That is, it allowed modification of the number of cycles for the on state of the signal line for both a 1 and a 0 bit but not of the remaining off time. The result was that my overall time per bit was longer than specification for the WS2812B based Neopixels (1.25 microseconds). With some modifications to the logic, it worked properly.

The next issue is a latent defect when the number of pixels in a string being driven is an even power of two. The logic that tests for when the last pixel was transmitted will fail and it looks forever leaving the pixels frozen with their initial value. This is due to a counter set up to hold the count of pixels, origin zero, thus in my case of 512 pixels, the counter runs from 0 to 511 and needs only 9 bits to hold that value. 

The logic tests to see when the current pixel is not less than the configured count, thus testing that the counter is not less than 512. It will always be less than 512 because it rolls over from 511 to 0 due to the counter width. I don't need all 512 pixel positions thus I will configure this for 510 which should work fine with the current logic.