Cyclone V SoC - Shared Memory Controller

--- Quote Start ---

Dear BadOmen,

As I have seen, you have a lot of valuable answers in the topic of Cyclone V SoC Development kit on the Altera Forum and on the rocketboards.org.

I will describe here my achievements and the problems:

-About 3-4 month ago I am working with Cyclone V Soc boards (formerly revA, now rev.C ES board).

FPGA portion usage:

-I have managed to reach and test the DDR3 memory region clearly within FPGA portion by a Nios II soft core, and a video-test works in 1080p. Nios II can reach and test/validate the memory in soft mode.

HPS portion usage:

-I have managed to reach and test the DDR3 memory region within HPS portion: ARM cores can reach and test/validate the memory in HPC mode. I modified the design from Nios II generated SimpleMemoryTEst template to ARM application processor, by using SOCAL-HWLib level of functions.

- Mixed mode usage: based on Golden SW/HW reference I can easily reach different GPIOs via LW HPS2FPGA bridge by using SOCAL-HWLib level of functions.

.

However, I have the following problems with mixed mode memory accesses (I intended to use HWLib level at first):

- How I can reach FPGA's DDR3 memory / internal BRAM from ARM side (when I set HPS2FPGA bridge)?

- How I can reach HPS' DDR3 memory from FPGA/Nios II side (when I set FPGA2HPS bridge)?

- Are there any reference design for such a mixed mode data transfers between HPS and FPGA portions by using DMA and without DMA conrtoller?

Unfortunately, still I cannot get any valuable response about this topic from Altera SRs.

Thank your for your answer in advance.

Best regards,

Zs.V.

--- Quote End ---

When accessing memory in the FPGA from the HPS you do so through the HPS-to-FPGA bridge. In order for those transactions to occur the following must take place:

1) There must be a valid FPGA image already configured in the FPGA fabric

2) The bridge must be supplied with a running clock

3) The bridge must be taken out of reset

4) The bridge must be mapped into the MPU memory space by enabling the appropriate remapping bit in the L3 interconnect GPV (by default that slave is not reachable)

When accessing memory in the HPS from the FPGA fabric you do so through either the FPGA-to-HPS bridge or the FPGA-to-SDRAM interface. In the case of accessing memory through the bridge all that stuff I said previously about the opposite direction still applies. Another thing to watch out for is memory span. The FPGA-to-HPS bridge and FPGA-to-SDRAM interface expose slave interfaces with 32 bits of addressing. Soft IP such as the Nios II processor only have 32-bit addresses so if you connect it to one of the HPS interfaces it will not have address bits for anything else in the system. So I would place the "Address Span Expander" bridge between your FPGA masters and the HPS slave ports which will expose a smaller window into the HPS address range. Another thing that you should do is for any Avalon-MM IP you connect to the HPS make sure you set the slave secruity accordingly. Avalon-MM doesn't support the AXI secruity signals and by default the HPS memories/peripherals are boot secure. So if you leave the default secruity alone in your system then you'll need to make sure those Avalon-MM masters in the FPGA fabric send secure transactions into the HPS. If you right click the column titles in the Qsys view you'll see a new secruity column is selectable, make sure to set the secruity accordingly.

The golden hardware reference design up on rocket boards shows some of what I discussed above but there are no DMAs present in the design. Somewhere on rocket boards there should be a design with a PCIe controller and some DMA engines that you can also take a look at. At a later time there should be more designs that show the type of interactions between the HPS and FPGA fabric that you are looking for so stay tuned for those.

Dear BadOmen,

Thank you for your answer.

I will try it and give some feed-back.

Regards,

ZS.V.

--- Quote Start ---

Also so that you don't spend time debugging a known issue, there is a known issue with the throughput between the FPGA and the HPS. This issue will be corrected in the next update.

--- Quote End ---

Can you elaborate more on this? I have some problems with the throughput. Are you referring to an update of Quartus, EDS, or the kernel?

Dear BadOmen,

Here is a feedback for your comments as I promised.

According to your previous post I have also managed to access now the FPGA SDRAM / internal BRAM from the HPS portion (ARM side) at HWLib level. The key was to release the bridges from reset (init_bridge in hwlib.c).

However, without DMA I have measured only 10 - 15 MB/s memory throughput (between ARM vs. FPGA memory) across HPS2FPGA bridge. Do you suggest any reference design, which integrates (FW/SW) and uses DMA?

What is the role the h2f_reset signal in HPS (cylcone V handbook gives a few info)? Because as I experienced if I left this signal unconnected in qsys I can use both FPGA memory and HPS memory in a same FW design? The debounce circuit is also eliminated from golden HW/SW reference design.

"Address Span Expander" bridge will be a good IP in the future, but I set all memory address spaces to a narrower size in order to be accessed by a 32-bit addresses of Nios II processor.

Regards,

ZS.V.

I don't know of any designs that DMA data between the FPGA fabric and HPS using the DMA controller built into the HPS. Up on Rocketboards there should be a PCIe design that DMAs data using a soft DMA controll in the FPGA between the two FPGAs over PCIe. If you don't require a lot of throughput then you can use the HPS DMA controller. If you require a lot of throughput then a soft DMA in the FPGA fabric will make more sense since it will not cycle through multiple channels like the HPS DMA does.

The h2f_reset is provided so that you can trigger a reset into the FPGA fabric from the HPS. It is optional to use but typically if you reset the HPS you'll want to reset FPGA peripherals connected to the HPS as well.

The reason why I brought up the address span expander is because the Nios II masters only provide 32-bits of addressing. So if you connect Nios II to any of the HPS slave ports you won't have any address bits left over to connect Nios II to anything else. That's why the bridge between the Nios II master and HPS is necessary.

It seems Voroshazi and I are trying to do a very similar thing. I have also been working on moving data from the HPS to the FPGA SDRAM across the HPS2FPGA bridge. I too have been able to achieve this but have the same issue in that the throughput I am getting is in the order of 58MB/sec (by memcpy from HPS SDRAM to FPGA SDRAM). I'm not using a DMA either and wondered if you had been able to achieve better than this (I need at least 80MB/sec for my application)?

does anybody have any insight as to why the data rate is so low? I was expecting it to be much faster.

Since I'm more of a hardware person my approach to this would be to have a soft DMA sitting in the FPGA that accesses the FPGA-to-SDRAM interface to move the data. I'm not sure what your end goal will be but is it possible for the FPGA to master the data directory by accessing the FPGA-to-SDRAM interface without the copy?

Based on the numbers you are seeing it sounds like memcpy doesn't make use of the DMA in the HPS. Even if it did use the HPS DMA, I would expect much more bandwidth to be possible if the FPGA read the data directly from the FPGA-to-HPS interface though.

--- Quote Start ---

Since I'm more of a hardware person my approach to this would be to have a soft DMA sitting in the FPGA that accesses the FPGA-to-SDRAM interface to move the data. I'm not sure what your end goal will be but is it possible for the FPGA to master the data directory by accessing the FPGA-to-SDRAM interface without the copy?

Based on the numbers you are seeing it sounds like memcpy doesn't make use of the DMA in the HPS. Even if it did use the HPS DMA, I would expect much more bandwidth to be possible if the FPGA read the data directly from the FPGA-to-HPS interface though.

--- Quote End ---

Hi BadOmen,

I have tried to build a simple FPGA block that interrogates the SDRAM on the HPS via the F2H SDRAM port.

I'm using a module from an example on rocketboards (http://www.rocketboards.org/foswiki/view/projects/myfirstmodule) that allocates a section of SDRAM on the HPS for the purposes of reading/writing via the FPGA. However on the FPGA side I cannot get a response to requests for memory reads, when viewed in signaltap, the waitrequest signal of the Avalon_MM bus is always high.

Do you know the reason for this and the solution? Is it possible that I need to do something in software to enable the F2H port on the HPS SDRAM.

--- Quote Start ---

Since I'm more of a hardware person my approach to this would be to have a soft DMA sitting in the FPGA that accesses the FPGA-to-SDRAM interface to move the data. I'm not sure what your end goal will be but is it possible for the FPGA to master the data directory by accessing the FPGA-to-SDRAM interface without the copy?

Based on the numbers you are seeing it sounds like memcpy doesn't make use of the DMA in the HPS. Even if it did use the HPS DMA, I would expect much more bandwidth to be possible if the FPGA read the data directly from the FPGA-to-HPS interface though.

--- Quote End ---

Hi BadOmen,

I have tried to build a simple FPGA block that interrogates the SDRAM on the HPS via the F2H SDRAM port.

I'm using a module from an example on rocketboards (http://www.rocketboards.org/foswiki/view/projects/myfirstmodule) that allocates a section of SDRAM on the HPS for the purposes of reading/writing via the FPGA. However on the FPGA side I cannot get a response to requests for memory reads, when viewed in signaltap, the waitrequest signal of the Avalon_MM bus is always high.

Do you know the reason for this and the solution? Is it possible that I need to do something in software to enable the F2H port on the HPS SDRAM.