PCIe core MSI-X support

--- Quote Start ---

MSI-X interrupt simply by issueing a single DW PCIe Memory Write to the appropriate MSI-X address with the appropriate data, as specified in the vector table.

--- Quote End ---

Memory Write address is 32 bit or 64 bit?

I don’t think your understanding of the location of the MSI-X vector table is correct.

The vector table is not placed inside the PCIe IP (hard or soft). Instead, the application has to present it behind the application interface (probably Avalon ST), and the values you enter in the MegaWizard for the address offsets and BARs denote your application’s expectations of those accesses.

Your application must be aware of the data written by the OS/driver to the given address ranges anyway, as it needs it for sending out the proper MSI-X messages.

The only job that the MegaWizard will do is implement the MSI-X Capability Structure including the control registers, so that your device will be recognized by the OS/driver as MSI-X capable and controllable. Before you are allowed to issue MSI-X writes, you have to make sure you are allowed to do so by polling cfg_msixcsr. This is done by the OS only when all entries of the MSI-X table have been written, i.e. are known to the application so that successive MSI-X write accesses can hit the right target (interrupt number).

--- Quote Start ---

The vector table is not placed inside the PCIe IP (hard or soft). Instead, the application has to present it behind the application interface (probably Avalon ST), and the values you enter in the MegaWizard for the address offsets and BARs denote your application’s expectations of those accesses.

--- Quote End ---

Yes. I'm get one megabyte in PCI memory and set MSI table to 0x80000. I'm make four 32bit registers for data and address ({32_bit_adress_low,32_bit_data}). But this is only low 32 bit of address. MSI-X required full 64 bit address. For generate IRQ I need generate memory write with 64bit address {32_bit_adress_high,32_bit_adress_low} with data in "32_bit_data".

Thank for all.

Now my MSI-X table for two entry looks line:

msix0_addr = msix1_addr = FFFFFFFFFFF0F00F // I test it on 64 bit Linux.

msix0_ctrl = msix1_ctrl = FFFFFFFE // LSB is 0, all ok.

msix0_data = FFFFFFA1

msix1_data = FFFFFFA9

looks good too.

I'm get msixcsr data from tl_cfg bus.

cfg_msixcsr[15] - MSI-X Enable – good.

cfg_msixcsr[14] - 0, “each vector’s Mask bit determines whether the

vector is masked or not. and msi_ctrl[0] i have 0.” Yes. I have.

I'm make memory write.

TLP_FMT_4DW_W, TLP_TYPE_WRITE, Last BE = 4'b0000, First BE = 4'b1111.

Adress is FFFFFFFFFFF0F00F.

But no interrupt.

looks very strange LSB in FFFFFFFFFFF0F00F. It's must be 2'b00.

The address looks strange indeed. Can you trap the write accesses to these registers and look what address Linux actually writes, on the AST level (as write data)? Is your MSI-X Table handler prepared for all kinds of write accesses, especially if they are aggregated?

Remember: The MSI-X Table layout specifies for Message Address bits 01:00: «For proper DWORD alignment, software must always write zeroes to these two bits; otherwise the result is undefined. The state of these bits after reset must be 0. These bits are permitted to be read only or

read/write.»

--- Quote Start ---

The address looks strange indeed. Can you trap the write accesses to these registers and look what address Linux actually writes, on the AST level (as write data)? Is your MSI-X Table handler prepared for all kinds of write accesses, especially if they are aggregated?

--- Quote End ---

I'm support no more than 64bit burst. Because I'm read x86 CPU can't generate burst bigger. I will check. Other's registers works ok.

--- Quote Start ---

Remember: The MSI-X Table layout specifies for Message Address bits 01:00: «For proper DWORD alignment, software must always write zeroes to these two bits; otherwise the result is undefined. The state of these bits after reset must be 0. These bits are permitted to be read only or

read/write.»

--- Quote End ---

Yes.

--- Quote Start ---

You might be curious to read intel’s apic spec regarding msi (http://www.intel.com/assets/pdf/manual/253668.pdf) (see Chapter 10.11) to get a feeling about valid values of the MSI/MSI-X message address and data.

--- Quote End ---

Thank! It's help.

Now MSI-X is working.