Discovered that I may not need the onchip RAM approach at all

MY ORIGINAL ISSUE AND ASSUMPTION

The original challenge I saw was that the Linux program in the HPS side is driving the timing of the read and write of the file contents over to the FPGA, specifically for the unload operation at the end of using a virtual 2315 disk cartridge. 

My assumption was that there was no straightforward way for the HPS side to know that the SDRAM contents had been read in the FPGA and were ready to be fed to the HPS side when it does the read. The H2F bridge picks the cycle when the read command signal will arrive at the FPGA.   

When we do an unload of the updated contents of the SDRAM back to the Linux file, we have the master issuing reads to get the next word (actually groups of four) but in the FPGA we might not have successfully completed the read of the word (four words) from SDRAM yet. The HPS side has no visibility into whether our read over the F2SDRAM channel has completed yet.

As a result, I was going to set up a complex system making use of 64K words of embedded RAM in the FPGA. At the start of an unload operation, the FPGA would read 64K words from SDRAM into this buffer, then signal to the HPS side that it is ready to deliver 64K words when the H2F channel issues those reads. The HPS side then asks for the next 64K block of words, we preread them from SDRAM, we indicate that we are ready, and the HPS side issues another 64K words worth of reads. This continues for the eight groups of 64K words needed to unload the entire disk cartridge image from SDRAM. 

The way we signal back and forth with the HPS side is via the H2F LW bridge which is used to send a command word to the FPGA. When that word is Unload, we start the preread of the first 64K words into the embedded RAM. The FPGA side then sets up a status word indicating we are ready for the HPS side to read our words - this is interrogated over the H2F LW channel when the HPS side reads that command/status word. 

I added the embedded RAM because the alternate of using the H2F LW command/status word to synchronize every single word of the unload would involve over a half a million such protocol exchanges. Still, it is a bit of a Rube Goldberg/Heath Robinson sort of mechanism. 

WAITREQUEST SIGNAL CAN HOLD OFF THE MASTER

Part of the memory mapped Avalon protocol used to read and write on the H2F LW, H2F, and F2H bridge channels is a signal called waitrequest which I suddenly realized was the much simpler mechanism that I needed to solve the synchronization issue. It is a standard part of the protocol and by controlling that signal, I could make the HPS side wait when it issued a read for a word, so that I could complete the read from SDRAM before deasserting waitrequest to let the HPS side read proceed. 

Showing waitrequest delaying the read

This does stall the channel, as the controlling side of the channel in HPS will remain active waiting for its read operation to complete. If I pause a long time or never release the waitrequest, the controlling side (M side) remains busy. This could be a problem if that H2F channel were connected to multiple mechanisms in my FPGA which independently could be attempting to communicate, but I can control this. 

Thus, I will go back to a simple loop for all 521,304 words which is 130, 326 times when I read a group of four words from SDRAM and then release waitrequest so that the read of those four words from the HPS side can go forward. If I happen to complete the read from SDRAM before the HPS side does a read of the H2F channel, all is good and it immediately satisfies the read by HPS. If I am not done yet, the HPS side read will hang as long as waitrequest remains asserted. 

That is a much simpler and more straightforward way to handle the unload process, relieving the need for a ballet of command words and status signals over the H2F LW channel and eliminating the need for advance reading into embedded RAM. I am pleased with this approach.