Forum Discussion
Honored ContributorPCIe core MSI-X support
I have added MSI-X support to an implementation of the Altera PCIe core we're already testing in the lab. My understanding, is that with MSI-X enabled in the megawizard and my vector table offset set appropriately, I have created the vector table in my own register set at that offset. I then, as I understand it, trigger an MSI-X interrupt simply by issueing a single DW PCIe Memory Write to the appropriate MSI-X address with the appropriate data, as specified in the vector table. As I understand it, the specific MSI I/O to the core isn't used for MSI-X, and you just do a normal memory write.
However, my write doesn't seem to appear in shared memory in the simulation. I'm not sure what I'm doing wrong. Is my above understanding correct? I have other data moving back and forth over the link without trouble. I notice that in the shared memory the code 0x00207811 appears, in the first interrupt address, even with no interrupt happening, image attached. Also attached is a wave view of my MSI-X single DW write.16 Replies
Altera_Forum--- Quote Start --- MSI-X interrupt simply by issueing a single DW PCIe Memory Write to the appropriate MSI-X address with the appropriate data, as specified in the vector table. --- Quote End --- Memory Write address is 32 bit or 64 bit?- Altera_Forum
I don’t think your understanding of the location of the MSI-X vector table is correct.
The vector table is not placed inside the PCIe IP (hard or soft). Instead, the application has to present it behind the application interface (probably Avalon ST), and the values you enter in the MegaWizard for the address offsets and BARs denote your application’s expectations of those accesses. Your application must be aware of the data written by the OS/driver to the given address ranges anyway, as it needs it for sending out the proper MSI-X messages. The only job that the MegaWizard will do is implement the MSI-X Capability Structure including the control registers, so that your device will be recognized by the OS/driver as MSI-X capable and controllable. Before you are allowed to issue MSI-X writes, you have to make sure you are allowed to do so by polling cfg_msixcsr. This is done by the OS only when all entries of the MSI-X table have been written, i.e. are known to the application so that successive MSI-X write accesses can hit the right target (interrupt number). - Altera_Forum--- Quote Start --- The vector table is not placed inside the PCIe IP (hard or soft). Instead, the application has to present it behind the application interface (probably Avalon ST), and the values you enter in the MegaWizard for the address offsets and BARs denote your application’s expectations of those accesses. --- Quote End --- Yes. I'm get one megabyte in PCI memory and set MSI table to 0x80000. I'm make four 32bit registers for data and address ({32_bit_adress_low,32_bit_data}). But this is only low 32 bit of address. MSI-X required full 64 bit address. For generate IRQ I need generate memory write with 64bit address {32_bit_adress_high,32_bit_adress_low} with data in "32_bit_data".
- Altera_Forum
Look at the other thread, I gave you a link to the correct MSI-X description (here it is again (http://www.pcisig.com/specifications/conventional/msi-x_ecn.pdf)). Your description is most probably from the PCI 3.0 pre-release which incorporates an early version of msi-x. I have that pre-release, too, so I understand your confusion :)
- Altera_Forum
Thank for all.
Now my MSI-X table for two entry looks line: msix0_addr = msix1_addr = FFFFFFFFFFF0F00F // I test it on 64 bit Linux. msix0_ctrl = msix1_ctrl = FFFFFFFE // LSB is 0, all ok. msix0_data = FFFFFFA1 msix1_data = FFFFFFA9 looks good too. I'm get msixcsr data from tl_cfg bus. cfg_msixcsr[15] - MSI-X Enable – good. cfg_msixcsr[14] - 0, “each vector’s Mask bit determines whether the vector is masked or not. and msi_ctrl[0] i have 0.” Yes. I have. I'm make memory write. TLP_FMT_4DW_W, TLP_TYPE_WRITE, Last BE = 4'b0000, First BE = 4'b1111. Adress is FFFFFFFFFFF0F00F. But no interrupt. looks very strange LSB in FFFFFFFFFFF0F00F. It's must be 2'b00. - Altera_Forum
The address looks strange indeed. Can you trap the write accesses to these registers and look what address Linux actually writes, on the AST level (as write data)? Is your MSI-X Table handler prepared for all kinds of write accesses, especially if they are aggregated?
Remember: The MSI-X Table layout specifies for Message Address bits 01:00: «For proper DWORD alignment, software must always write zeroes to these two bits; otherwise the result is undefined. The state of these bits after reset must be 0. These bits are permitted to be read only or read/write.» - Altera_Forum--- Quote Start --- The address looks strange indeed. Can you trap the write accesses to these registers and look what address Linux actually writes, on the AST level (as write data)? Is your MSI-X Table handler prepared for all kinds of write accesses, especially if they are aggregated? --- Quote End --- I'm support no more than 64bit burst. Because I'm read x86 CPU can't generate burst bigger. I will check. Other's registers works ok. --- Quote Start --- Remember: The MSI-X Table layout specifies for Message Address bits 01:00: «For proper DWORD alignment, software must always write zeroes to these two bits; otherwise the result is undefined. The state of these bits after reset must be 0. These bits are permitted to be read only or read/write.» --- Quote End --- Yes.
- Altera_Forum
btw, I don’t know why msix*_ctrl has so many '1's in it. Those bits should be cleared at reset, and the OS should not write anything beside zero into it, anyway.
- Altera_Forum
You might be curious to read intel’s apic spec regarding msi (http://www.intel.com/assets/pdf/manual/253668.pdf) (see Chapter 10.11) to get a feeling about valid values of the MSI/MSI-X message address and data.
- Altera_Forum--- Quote Start --- You might be curious to read intel’s apic spec regarding msi (http://www.intel.com/assets/pdf/manual/253668.pdf) (see Chapter 10.11) to get a feeling about valid values of the MSI/MSI-X message address and data. --- Quote End --- Thank! It's help. Now MSI-X is working.
