Forum Discussion

BInos's avatar
BInos
Icon for New Contributor rankNew Contributor
6 years ago

Cyclone IV GX Platformdesigner PCIe Hard-IP Upstream Port Arbitration

Hello!

I am using a Cyclone IV GX FPGA with the integrated PCIe Hard-IP.

The FPGAs PCIe is connected to an ARMv7 CPU running Linux.

In the FPGA Design (with Platfromdesigner) the PCIe Hard-IP's Avalon txs Slave Port and multiple Avalon Master Bars are used.

Two DMA controllers inside the FPGA write Data to the CPUs RAM via the txs Port. And the CPU has read/write access to the Avalon Slaves via the bars.

Now i have the problem of arbitration/priorities between txs and bar ports. After reading the IP Compiler for PCIe Userguide, I still have no clue which port has a higher priority (also priorities between different bars would be interessting).

In my application the txs should have the lowest priority, so that the DMAs get throttled. As result a low latency read/write to bars should be possible.

I read about PCIe's Virtual Channels, but as far as I know the Cyclone IV does only support one Virtual Channel, so no solution?

Is there any way to setup arbitration/priorities between PCIe txs and bars:

  • Using PCIe Configurationspace (Config of FPGA Endpoint / CPU Rootport?
  • Adding Arbitration mechanism in Platform Designer between txs and Bars?

Thanks in advance!

3 Replies

  • Nathan_R_Intel's avatar
    Nathan_R_Intel
    Icon for Contributor rankContributor
    6 years ago
    Hie, Please check my replies to all your questions. Question: Now i have the problem of arbitration/priorities between txs and bar ports. After reading the IP Compiler for PCIe Userguide, I still have no clue which port has a higher priority (also priorities between different bars would be interessting). Response: There is no priority specified between using Rx Master Module and TX Slave(TXS) Module. Hence, if you are using the AVMM bridge provided by the IP compiler, you can either use either the Rx Master or TXS to access the BAR addresses. The IP itself did not specify any prioritization between RX Master and TXS. Question: In my application the txs should have the lowest priority, so that the DMAs get throttled. As result a low latency read/write to bars should be possible. Response: Currently, the Cyclone IX GX PCIe Hard IP does not offer a feature to lower prioritize the txs. Instead, I will recommend to use the tx_ws signal which creates a DLLP transmission priority. Please check if this can be useful to your application. You may refer to Pg B-22 (Figure B-19) in following user guide for some reference on using the tx_ws signal. https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/ug/ug_pci_express.pdf Question: I read about PCIe's Virtual Channels, but as far as I know the Cyclone IV does only support one Virtual Channel, so no solution? Yes only one VC is supported. Question: Is there any way to setup arbitration/priorities between PCIe txs and bars: Using PCIe Configurationspace (Config of FPGA Endpoint / CPU Rootport? Adding Arbitration mechanism in Platform Designer between txs and Bars? Response: As explained above, one method is to use the tx_ws signal. Alternatively, you can use software to specify priority on servicing requested interrupts and use interrupts. As for using config space or arbitration on the interface to specify priority, this has not be implemented or tested using our Hard IP before. Hence, its feasibility and implementation challenges is unknown. Regards, Nathan
  • sadad's avatar
    sadad
    Icon for New Contributor rankNew Contributor
    6 years ago

    Thanks for your answers.

    Response:

    Currently, the Cyclone IX GX PCIe Hard IP does not offer a feature to lower prioritize the txs. Instead, I will recommend to use the tx_ws signal which creates a DLLP transmission priority. Please check if this can be useful to your application.

    You may refer to Pg B-22 (Figure B-19) in following user guide for some reference on using the tx_ws signal.

    https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/ug/ug_pci_express.pdf

    Followup:

    I am using the QSYS/Platformdesigner method. So i think i won't be able to use the tx_ws at all?

    Response:

    As explained above, one method is to use the tx_ws signal. Alternatively, you can use software to specify priority on servicing requested interrupts and use interrupts.

    As for using config space or arbitration on the interface to specify priority, this has not be implemented or tested using our Hard IP before. Hence, its feasibility and implementation challenges is unknown.

    Followup:

    Currently the design is using two datapaths:

    1. time critical: One datapath issues a MSI with an interval of x kHz => the isr handler fetches data from bar0.
    2. lower priority: Two DMA controllers in the qsys design write/read data to the host memory over the txs port and issue MSIs when they are finished.

    The issue is that reading data from the isr handler via bar0 gets delayed by the concurrently access of the DMA controllers. I also think that the MSIs requests will get delayed?

    How could tx_ws help me in this design?

  • Nathan_R_Intel's avatar
    Nathan_R_Intel
    Icon for Contributor rankContributor
    6 years ago
    Hie, I am afraid based on your use case, Cyclone IV GX might not have a solution to enable two data-paths with prioritization. tx_ws is basically using the application layer to create a wait state; hence I can't think of a way to meet your use case. Also, this data phase signals are not available in QSYS/Platform Editor environment. Regards, Nathan