Designing C code to write over the HPS to FPGA bridges

MEMORY MAPPING OF THE BRIDGE HARDWARE

The hardware on the Hard Processor System (HPS) side, including the bridges but also many other devices like ethernet controllers and SD Card controllers, are accessed by reading and writing to specific locations in memory. That is, the hardware is memory mapped. The overall address space for these ARM processor systems is 4GB, but ranges of addresses are carved out for the various peripherals. 

Our Cyclone V system on a chip  (SoC) provides four bridges between the HPS side and the Field Programmable Gate Array (FPGA) side. One of them is actually a bridge between the FPGA and the DRAM controller logic in the HPS system - the F2SDRAM bridge - which can directly access all of the SDRAM. The other three are the H2F, F2H and H2F LW (Lightweight) bridges.  

Only these last three are memory mapped into the processor address space since the first one only communicates with the DRAM controller. Looking at the three address ranges below, you see that the F2SDRAM has the rightmost map - addresses from 0 up to 4GB will directly access those same addresses in DRAM. Since this board has only 1GB of DRAM installed, the F2SDRAM bridge can only use addresses from 0 to 1GB.

Three addressing ranges

The middle address range is what the CPU sees, thus my programs (and Linux) have this view. The left address range is the one visible to the F2H bridge which we are not using in this project. The left and middle ones both have memory mapped the range of addresses from 3GB up to 4GB for access to the FPGA and peripherals.

The H2F bridge is accessed by touching addresses from the 3GB line xC0000000 up to the peripheral zone start at xFC000000. These are mapped down to addresses inside the FPGA, with the start of the area (xC000000 on the HPS side) appearing as x0000000 in the FPGA. Thus the H2F bridge has 960MB of addressible words that can be routed to various logic in the FPGA where the logic uses the address to determine if it is being selected. 

My logic in the FPGA implements a single four word buffer and ignores the address. Thus, any of the 960MB of addresses from x00000000 onwards will simply read or write this block of four words in the FPGA. From the HPS side, whether I write to xC0000000 or xE8000000 I am still reading or writing my four word block in the FPGA. 

The last 64MB of the address range is the peripheral area xFC000000 to xFFFFFFFF where all the peripherals and control registers are accessible by using addresses in this range. A 2MB slice of this peripheral region is assigned to the H2F LW bridge, the lightweight bridge. Akin to the other bridge, the address xFF200000 is the start of the H2FLW address range and these become addresses 0 to 2MB on the FPGA side. 

My logic in getcommands module treats any address in the 2MB range beginning at xFF200000 in the HPS side address range as the single command word. I could write to xFF2000000 or xFF200100 and it will be accepted by my getcommands module as a command or status word request. In the FPGA, I ignore the address, so whether it comes in as x00000000 or x00000100, using the hypothetical above, it is accepted to write the command word or read the status word. 

CONTROL REGISTERS FOR ACTIVATION OF EACH BRIDGE

The peripherals region implements 64 MB of various peripheral devices and control words. The configuration registers for the various bridges (H2F, F2H, H2FLW and F2SDRAM) are implemented at defined ranges. The H2F bridge control words begin at xFF500000 while the H2FLW control words start at xFF400000 and these are immediately following the end of the 2MB range of the H2FLW bridge.

Writing given values that set defined bits of defined words in the bridge configuration registers, called the Global Programmers View (GPV) in the documentation, to configure and activate that bridge. The F2SDRAM bridge does not have GPV registers nor need explicit activation, I believe, unlike the other three. 

ADDRESSING UNUSED FOR BOTH H2F AND H2FLW BRIDGES, ESSENTIAL FOR F2SDRAM

At the present time, with the two H2F bridges used for one purpose each, there is no need to discriminate the address for a read or write as we know it is always the one and only word or block of four words. The Avalon Memory Mapped protocol does provide an address but I was free to ignore it. 

The F2SDRAM interface, on the other hand, could access any word of the entire 1GB DRAM. Since almost all of that is in use from Linux, I have to carefully address each transaction to point only at the top 1MB which I had reserved for my use. The logic in loopcart and in diskmodel will produce addresses only in that last megabyte range. 

C CODE THAT WRITES COMMANDS AND FETCHES STATUS WORDS

The basics of accessing the bridge from Linux is to point at the proper address in the range that is associated with the give bridge or GPV register. To do this, we have to get access to the memory range which is done by using a special file, /dev/mem and then memory mapping it with a Linux system call. We start at the predefined addresses - xC0000000 for the H2F bridge where we will read/write the blocks of four words to the FPGA, and xFF20000 for the H2FLW bridge where we will write the command word and read the status word from FPGA. The GPV registers for those bridges begin at xFF400000 and xFF500000.

The resulting addresses for the H2F data, the H2FLW data and the GPV control registers are pointers that we use to load and store from the C program. Assuming we have the two bridges configured and operational, then in order to send a command that begins a load operation using loopcart, we set the last four bits of a word to the code b0001 that represents the load command. 

That word is then stored through a pointer to location xFF200000 and automagically the bridge hardware issues a write on the H2FLW bridge. My logic in the FPGA then triggers loopcart which begins to receive blocks of four words from H2F and writes them to the SDRAM. Checking through the pointer generates a read transaction where my logic sends the status word to the HPS side. 

C CODE TO PERFORM A LOAD OPERATION

Once the command word is issued to the FPGA to begin a load operation, the FPGA side is waiting for writes from us on the H2F bridge. We will open a virtual 2315 cartridge image file, memory map it, and then transfer blocks of four words through another pointer (to xC0000000) which forces a write transaction. Our code must loop through the entire cartridge file, four words at a time, for a total of 130,326 times to load the entire cartridge file into the last megabyte of SDRAM.