I have added MSI-X support to an implementation of the Altera PCIe core we're already testing in the lab. My understanding, is that with MSI-X enabled in the megawizard and my vector table offset set appropriately, I have created the vector table in my own register set at that offset. I then, as I understand it, trigger an MSI-X interrupt simply by issueing a single DW PCIe Memory Write to the appropriate MSI-X address with the appropriate data, as specified in the vector table. As I understand it, the specific MSI I/O to the core isn't used for MSI-X, and you just do a normal memory write. However, my write doesn't seem to appear in shared memory in the simulation. I'm not sure what I'm doing wrong. Is my above understanding correct? I have other data moving back and forth over the link without trouble. I notice that in the shared memory the code 0x00207811 appears, in the first interrupt address, even with no interrupt happening, image attached. Also attached is a wave view of my MSI-X single DW write.

--- Quote Start --- MSI-X interrupt simply by issueing a single DW PCIe Memory Write to the appropriate MSI-X address with the appropriate data, as specified in the vector table. --- Quote End --- Memory Write address is 32 bit or 64 bit?

I don’t think your understanding of the location of the MSI-X vector table is correct. The vector table is not placed inside the PCIe IP (hard or soft). Instead, the application has to present it behind the application interface (probably Avalon ST), and the values you enter in the MegaWizard for the address offsets and BARs denote your application’s expectations of those accesses. Your application must be aware of the data written by the OS/driver to the given address ranges anyway, as it needs it for sending out the proper MSI-X messages. The only job that the MegaWizard will do is implement the MSI-X Capability Structure including the control registers, so that your device will be recognized by the OS/driver as MSI-X capable and controllable. Before you are allowed to issue MSI-X writes, you have to make sure you are allowed to do so by polling cfg_msixcsr. This is done by the OS only when all entries of the MSI-X table have been written, i.e. are known to the application so that successive MSI-X write accesses can hit the right target (interrupt number).

--- Quote Start --- The vector table is not placed inside the PCIe IP (hard or soft). Instead, the application has to present it behind the application interface (probably Avalon ST), and the values you enter in the MegaWizard for the address offsets and BARs denote your application’s expectations of those accesses. --- Quote End --- Yes. I'm get one megabyte in PCI memory and set MSI table to 0x80000. I'm make four 32bit registers for data and address ({32_bit_adress_low,32_bit_data}). But this is only low 32 bit of address. MSI-X required full 64 bit address. For generate IRQ I need generate memory write with 64bit address {32_bit_adress_high,32_bit_adress_low} with data in "32_bit_data".

Look at the other thread, I gave you a link to the correct MSI-X description (here it is again (http://www.pcisig.com/specifications/conventional/msi-x_ecn.pdf)). Your description is most probably from the PCI 3.0 pre-release which incorporates an early version of msi-x. I have that pre-release, too, so I understand your confusion :)

Thank for all. Now my MSI-X table for two entry looks line: msix0_addr = msix1_addr = FFFFFFFFFFF0F00F // I test it on 64 bit Linux. msix0_ctrl = msix1_ctrl = FFFFFFFE // LSB is 0, all ok. msix0_data = FFFFFFA1 msix1_data = FFFFFFA9 looks good too. I'm get msixcsr data from tl_cfg bus. cfg_msixcsr[15] - MSI-X Enable – good. cfg_msixcsr[14] - 0, “each vector’s Mask bit determines whether the vector is masked or not. and msi_ctrl[0] i have 0.” Yes. I have. I'm make memory write. TLP_FMT_4DW_W, TLP_TYPE_WRITE, Last BE = 4'b0000, First BE = 4'b1111. Adress is FFFFFFFFFFF0F00F. But no interrupt. looks very strange LSB in FFFFFFFFFFF0F00F. It's must be 2'b00.

PCIe core MSI-X support | Altera Community

16 Replies

Altera_Forum
Honored Contributor
14 years ago
Hello! It's seems that I'm made everything ok and interrupts came to CPU. But problem appear - MSI-X is working only on 1 PC (I have tried on other PC and on 2 servers). I saw on Signal TAP that MSI-X write-packet is formed exactly like on 1st PC...
Some additional information:
1. I'm using Arria II GX and PCIe Hard IP.
2. MSI interrupt are ok on all computers
3. All other write packets (not MSI-X) are also ok on all
4. Format of the MSI-X packet
assign Sig_Packet_Type_field = 8'h60;
assign Sig_Packet_Length_field = 10'b00_0000_0001;
Clk1: PCIe_data_out [63:0]<= {Requester_ID, 16'h00_0f, Sig_Packet_Type_field, 14'b0, Sig_Packet_Length_field};
Clk2: PCIe_data_out [63:0]<= {msix_Msg_Lower_Addr[msix_Number],msix_Msg_Upper_Addr[msix_Number]}
Clk3: PCIe_data_out [63:0]<= {Sig_Packet_Data_field, 32'b0};
Altera_Forum
Honored Contributor
14 years ago
You know that you must use a 3DW header if msix_Msg_Upper_Addr[msix_Number] is all zeroes?
Altera_Forum
Honored Contributor
14 years ago
Do You mean 3-DWord Header TLP and memory write request 32bit addr?
So how should packet look like if msix_Msg_Upper_Addr[msix_Number] is all zeroes?
Like this - only two clk:
assign Sig_Packet_Type_field = 8'h40; // 32bit addr
assign Sig_Packet_Length_field = 10'b00_0000_0001; // 1 DWORD
// LastBE = 4'h0, FirstBE = 4'hF
Clk1: PCIe_data_out [63:0]<= {Requester_ID, 16'h00_0f, Sig_Packet_Type_field, 14'b0, Sig_Packet_Length_field};
Clk2: PCIe_data_out [63:0]<= {msix_Msg_Lower_Addr[msix_Number], Sig_Packet_Data_field [31:0]}

Or like this – 3 clk:
assign Sig_Packet_Type_field = 8'h40; // 32bit addr
assign Sig_Packet_Length_field = 10'b00_0000_0001; // 1 DWORD
// LastBE = 4'h0, FirstBE = 4'hF
Clk1: PCIe_data_out [63:0]<= {Requester_ID, 16'h00_0f, Sig_Packet_Type_field, 14'b0, Sig_Packet_Length_field};
Clk2: PCIe_data_out [63:0]<= {msix_Msg_Lower_Addr[msix_Number],32'b0}
Clk3: PCIe_data_out [63:0]<= {Sig_Packet_Data_field, 32'b0};

It seems to me that I tried already both variants but they don't work :( HELP! :)
P.S. I checked msix_Msg_Upper_Addr is all 0!
On 1st PC where MSI-X are working, I send 4DW header and msix_Msg_Upper_Addr still is all 0...
Altera_Forum
Honored Contributor
14 years ago
--- Quote Start ---
Do You mean 3-DWord Header TLP and memory write request 32bit addr?
[…]
Or like this – 3 clk:
[…]

--- Quote End ---

Yes, either the first one (if addr mod 8 == 4) or the second one (if addr mod 8 == 0).

--- Quote Start ---

It seems to me that I tried already both variants but they don't work :(

--- Quote End ---

So it’s probably something else, additionally. Does the system properly enable MSI-X at all? There are systems that do support MSI but don’t support MSI-X.
Altera_Forum
Honored Contributor
14 years ago
--- Quote Start ---
Yes, either the first one (if addr mod 8 == 4) or the second one (if addr mod 8 == 0).
--- Quote End ---

I made in this way... but problem was that in 2clk packet I send
Clk2: PCIe_data_out [63:0]<= {msix_Msg_Lower_Addr[msix_Number], Sig_Packet_Data_field [31:0]}
but should be
Clk2: PCIe_data_out [63:0]<= {Sig_Packet_Data_field [31:0],msix_Msg_Lower_Addr[msix_Number]}

Thank You for help!!!! :)
Now everything is ok.
P.S. 1st PC understand write packet with 64 bit addr even if upper addr is all 0... but other don't understand... (they understand just 32bit addr packets)
Altera_Forum
Honored Contributor
14 years ago
Congratulations! ;)

Forum Discussion

PCIe core MSI-X support

16 Replies

Recent Discussions

Cyclone IV GX project failed migration from 20.1 to 23.1std

Cyclone 10 GX IBIS-AMI models

PCIe Enumeration Failure for CXL IP

Can't generate F-Tile Ethernet Hard IP Design Example

Agilex 7 slew rate reconfiguration