Forum Discussion

Mikhail_a's avatar
Mikhail_a
Icon for Occasional Contributor rankOccasional Contributor
21 days ago

Agilex-7 MSI-X missing om AXI MCDMA for PCIe

Hi!

We are using AGIB023R18A1E1VC device and AXI MCDMA for PCIe IP core. We set it to MCDMA+BAS+BAM mode as we want to use MSI-X interrupts. Our driver register for example 2 MSI-X interrupts in Linux with no issues, after that FPGA logic sends request to user_msix interface and observes handshake. But then we see no interrupt happening in host, and interrupts counter stays at zero value. 

In lspci we see that MSI-X capability exists and the number of MSI-X vectors is not zero.

Capabilities: [b0] MSI-X: Enable+ Count=53 Masked-

Vector table: BAR=0 offset=00100000

PBA: BAR=0 offset=00180000

We use Q25.3.1. 

Could you please help us with understanding what may be wrong and why we don't see interrupts in our host system?

Thank you in advance.

Mikhail.

Pcie

7 Replies

  • Mikhail_a's avatar
    Mikhail_a
    Icon for Occasional Contributor rankOccasional Contributor
    15 days ago

    Hi Wincent_Altera​ 

    I'm still interested if I need to enable MCDMA somehow so it can handle MSI-X requests. Because from MCDMA control regiters description it is not that obvious if we need to do something for MSI-X to be sent. 

  • Mikhail_a's avatar
    Mikhail_a
    Icon for Occasional Contributor rankOccasional Contributor
    15 days ago

    Hi Wincent_Altera​ 

    As I undrestand settings for MSI interrupt and MSI-X interrupts are not shared. So i'm not sure if we can refer to that setting for MSI-X. 

    In our design we use only PCIe core and some additional components like AXI briges and DDR controller. But the core logic is our own. But the PCIe subsustem was taken from generated example design unchanged.

    We run into the same issue with MCDMA for PCIe with Avalon buses. So it seems there is the same cause. 

    I just want to find out if we can take a look at some interface, so probably we will be able to see if write TLP for incoming MSI-X request is actually being sent, so we could localize the problem.

  • Mikhail_a's avatar
    Mikhail_a
    Icon for Occasional Contributor rankOccasional Contributor
    15 days ago

    Hi Wincent_Altera​ 

    At the moment we cant find anything wrong according to your list. Is there any way to make sure that

    1. our pcie endpoint actually sends MSI-X TLP to the root complex?
    2. the acual write transaction from the root complex was issued in the host system?

    Also one more question. As I understood from the doc MSI-X messages are written by MCDMA, but should we enable it or initialize if in our case we don't use it at all? 

    • Wincent_Altera's avatar
      Wincent_Altera
      Icon for Regular Contributor rankRegular Contributor
      15 days ago

      Hi Mikhail_a​ ,

      As I understood from the doc MSI-X messages are written by MCDMA, but should we enable it or initialize if in our case we don't use it at all? 
      >> If I refer to the user guide , the default value is 1. If you dont use it at all perhaps you can set it as zero

      our pcie endpoint actually sends MSI-X TLP to the root complex?
      the acual write transaction from the root complex was issued in the host system?
      >> Based on my understanding on PCIe architecture , the Endpoint generates MSI-X by sending a Memory Write TLP upstream. The Root Complex does not issue that write; it receives it and converts it into a CPU interrupt.

      What is the design you are using ? is it custom design ?
      Or design generate purely from our IP catalog ?

      If this is custom design, I would suggest to generate one from our IP catalog
      https://docs.altera.com/r/docs/847470/25.1.1/gts-axi-multichannel-dma-ip-for-pci-express-user-guide/bam-bas-mcdma-mode

      Regards,
      Wincent

  • Wincent_Altera's avatar
    Wincent_Altera
    Icon for Regular Contributor rankRegular Contributor
    15 days ago

    Hi,

    Is there anything else you think I can better assist ?

    Regards,
    Wincent

  • Wincent_Altera's avatar
    Wincent_Altera
    Icon for Regular Contributor rankRegular Contributor
    20 days ago

    Hi,

    Based on your description, you have correctly configured the hardware and Linux driver for MSI-X, and lspci confirms MSI-X is enabled with available vectors.
    However, the interrupts are not seen on the host. 

    I would suggest to check 

    • MSI-X table mapping
      • Ensure the BAR (Base Address Register) specified for the MSI-X vector table (e.g., BAR=0 at offset 0x00100000) is correctly mapped and accessible by both the FPGA and the Linux OS.
      • If the mapping is off, MSI-X table writes may not trigger interrupts.
    • Masked vectors
      • MSI-X must remain enabled (Enable+ in lspci).
      • Vectors must be unmasked (Masked- in lspci). If vectors are masked in the device or host, interrupts are blocked.
      • Confirm that the device or driver does not inadvertently mask vectors after initialization.
    • Driver initialization issues,
      • Maps the MSI-X vectors properly
      • Registers the interrupt handlers correctly
      • Binds the expected number of vectors (matching what is requested)

    Regards,
    Wincent