Forum Discussion

Altera_Forum's avatar
Altera_Forum
Icon for Honored Contributor rankHonored Contributor
14 years ago

How to request multiply MSI interrupts?

Hi !!!

Developing a network device driver on Arria2 GX (Avalon-ST).

Is it possible to set a vector of 8 MSI interrupts , if the design supports only MSI interrupts, not MSI - X?

From the side of device driver developer - to request 8 separate interrupt handlers, to handle 8 packet queues.

pci_enable_msi_block() returns 1.This means, that it's possible to request only 1 interrupt line.

The capabilities of device:

Capabilities: [78] Power Management version 3

Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)

Status: D0 PME-Enable- DSel=0 DScale=0 PME-

Capabilities: [80] Express (v1) Endpoint, MSI 00

DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us

ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-

DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-

RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+

MaxPayload 128 bytes, MaxReadReq 512 bytes

DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-

LnkCap: Port# 1, Speed 2.5GT/s, Width x4, ASPM L0s, Latency L0 unlimited, L1 unlimited

ClockPM- Suprise- LLActRep- BwNot-

LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+

ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-

LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-

Developing under Linux.

Best Regards,

Igor

39 Replies

Show More
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    13 years ago

    1. pci_enable_msi_block(). Will return irq numbers.

    2. request_irq . Pass irq numbers, which returned by pci_enable_msi_block().

    3. Check your interrupts . cat /proc/interrupts.

    In your irq handler , printk irq number, which you receive.

    Regards,

    Igor
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    13 years ago

    Hi,

    I finally get it with severals MSIs.

    Just take a look below: 

    
#dmesg
 pci 0000:04:00.0: irq 47-48 for MSI/MSI-X
# cat /proc/interrupts
            CPU0
   0:      49252   IO-APIC-edge      timer
   1:          2   IO-APIC-edge      i8042
    ........
   5:          0   IO-APIC-edge      parport0
   8:          1   IO-APIC-edge      rtc0
  46:       9454   PCI-MSI-edge      eth1
  47:          0   PCI-MSI-edge      LATTICE_pci

    Let's say 47 is my CAN

    and 48 is my UART 

    
Question 1
May I need to request twice as you sa
  if (request_irq (47,(irq_handler_t) irqreturn_CAN_isr, 
                   IRQF_DISABLED, LATTICE_PCI_NAME, &LATTICE_pci_dev))
  if (request_irq (48,(irq_handler_t)irqreturn_UART_isr, 
                   IRQF_DISABLED, LATTICE_PCI_NAME, &LATTICE_pci_dev))
--------------------Interrupt Handlers---------------------------------
 static irqreturn_CAN_isr ( 47, void *dev ) {
      void testCAN();
   
      printk(KERN_DEBUG " CAN received irq = %d \n", irq); 
   
      return IRQ_HANDLED;
   
  }
  
 static irqreturn_UART_isr ( int irq, void *dev ) {
      void testUART();
   
      printk(KERN_DEBUG " UART received irq = %d \n", irq); 
   
      return IRQ_HANDLED;
   
  }

    Is it OK?
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    13 years ago

    Hi,

    I have sucessfully test multiple MSI on my V6 Integrated Block Board and my concern now is:

    Do I need an MSI Controller to sychronise all the Interrupts source or it will be handle by the Core automatically.

    I would really appreciate any response in order to move forward.

    Regards,
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    13 years ago

    Hi, I need some advice for implementing MSI MaskRegister for my lattice Board which doesn't support MSI -X features(include Mask Register) I have implemented the MSI Interrupt and test it under Linux 1 months ago for my Lattice PCI Express IP Core with Wishbone interface for slaves. But now I am facing a problem to integrated a Mask register to facilitate the driver development. Below is what I have implemented . Could anybody give me a feedback to make sure I am on the rigth way? Notice: Lattice ECP3 just support up to 8 MSI Interrupts. module msi_maskregister( clk_i, rst_i, cyc_i, stb_i, adr_i, we_i, dat_i, dat_o, ack_o, msi ); parameter is = 8; // Number of interrupt sources // // Inputs & outputs // // 8bit WISHBONE bus slave interface input clk_i; // clock input rst_i; // reset (asynchronous active low) input cyc_i; // cycle input stb_i; // strobe (cycle and strobe are the same signal) input [ 2:1] adr_i; // address input we_i; // write enable input [ 7:0] dat_i; // data output output [ 7:0] dat_o; // data input output ack_o; // normal bus termination // Interrupt sources input [is:1] msi; // interrupt request inputs // // Module body // reg [is:1] pol, edgen, mask; // register bank reg [is:1] lirq, dirq; // latched irqs, delayed latched irqs // // latch interrupt inputs always @(posedge clk_i) lirq <=# 1 msi; // // generate delayed latched irqs always @(posedge clk_i) dirq <=# 1 lirq; // // generate actual triggers function trigger; input edgen, pol, lirq, dirq; reg edge_irq, level_irq; begin edge_irq = pol ? (lirq & ~dirq) : (dirq & ~lirq); ///////// Masking level_irq = pol ? lirq : ~lirq; trigger = edgen ? edge_irq : level_irq; end endfunction reg [is:1] irq_event; integer n; always @(posedge clk_i) for(n=1; n<=is; n=n+1) irq_event[n] <=# 1 trigger(edgen[n], pol[n], lirq[n], dirq[n]); // // generate wishbone register bank writes wire wb_acc = cyc_i & stb_i; // WISHBONE access wire wb_wr = wb_acc & we_i; // WISHBONE write access always @(posedge clk_i or negedge rst_i) if (~rst_i) begin pol <=# 1 {{is}{1'b0}}; // clear polarity register edgen <=# 1 {{is}{1'b0}}; // clear edge enable register mask <=# 1 {{is}{1'b1}}; // mask all interrupts end // 0x00: EdgeEnable Register // 0x01: PolarityRegister // 0x02: MaskRegister else if(wb_wr) // wishbone write cycle?? case (adr_i) // synopsys full_case parallel_case 2'b00: edgen <=# 1 dat_i[is-1:0]; // EDGE-ENABLE register 2'b01: pol <=# 1 dat_i[is-1:0]; // POLARITY register 2'b10: mask <=# 1 dat_i[is-1:0]; // MASK register endcase // // generate dat_o reg [7:0] dat_o; always @(posedge clk_i) case (adr_i) // synopsys full_case parallel_case 2'b00: dat_o <=# 1 { {{8-is}{1'b0}}, edgen}; //{ {{8-is}{1'b0}}, edgen} 2'b01: dat_o <=# 1 { {{8-is}{1'b0}}, pol}; 2'b10: dat_o <=# 1 { {{8-is}{1'b0}}, mask}; endcase // // generate ack_o reg ack_o; always @(posedge clk_i) ack_o <=# 1 wb_acc & !ack_o; endmodule

  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    13 years ago

    Hi, I need some advice for implementing MSI MaskRegister for my lattice Board which doesn't support MSI -X features(include Mask Register) I have implemented the MSI Interrupt and test it under Linux 1 months ago for my Lattice PCI Express IP Core with Wishbone interface for slaves. But now I am facing a problem to integrated a Mask register to facilitate the driver development. Below is what I have implemented . Could anybody give me a feedback to make sure I am on the rigth way? Notice: Lattice ECP3 just support up to 8 MSI Interrupts. 

    
module msi_maskregister(   clk_i, rst_i, cyc_i, stb_i, adr_i, we_i, dat_i, dat_o, ack_o,   msi );    parameter is = 8;            // Number of interrupt sources    //   // Inputs & outputs   //    // 8bit WISHBONE bus slave interface   input         clk_i;         // clock   input         rst_i;         // reset (asynchronous active low)   input         cyc_i;         // cycle   input         stb_i;         // strobe  (cycle and strobe are the same  signal)   input   adr_i;         // address   input         we_i;          // write enable   input   dat_i;         // data output   output  dat_o;         // data input   output        ack_o;         // normal bus termination    // Interrupt sources   input   msi;           // interrupt request inputs     //   //  Module body   //   reg   pol, edgen, mask;   // register bank   reg   lirq, dirq;         // latched irqs, delayed latched irqs     //   // latch interrupt inputs   always @(posedge clk_i)     lirq <=# 1 msi;    //   // generate delayed latched irqs   always @(posedge clk_i)     dirq <=# 1 lirq;     //   // generate actual triggers   function trigger;     input edgen, pol, lirq, dirq;      reg   edge_irq, level_irq;   begin       edge_irq  = pol ? (lirq & ~dirq) : (dirq & ~lirq);  /////////  Masking       level_irq = pol ? lirq : ~lirq;        trigger = edgen ? edge_irq : level_irq;   end   endfunction    reg   irq_event;   integer n;   always @(posedge clk_i)     for(n=1; n<=is; n=n+1)       irq_event <=# 1 trigger(edgen, pol, lirq, dirq);     //   // generate wishbone register bank writes   wire wb_acc = cyc_i & stb_i;                   // WISHBONE access   wire wb_wr  = wb_acc & we_i;                   // WISHBONE write  access    always @(posedge clk_i or negedge rst_i)     if (~rst_i)       begin           pol   <=# 1 {{is}{1'b0}};              // clear polarity  register           edgen <=# 1 {{is}{1'b0}};              // clear edge enable  register           mask  <=# 1 {{is}{1'b1}};              // mask all interrupts       end // 0x00: EdgeEnable Register // 0x01: PolarityRegister // 0x02: MaskRegister      else if(wb_wr)                               // wishbone write  cycle??       case (adr_i) // synopsys full_case parallel_case         2'b00: edgen <=# 1 dat_i;        // EDGE-ENABLE register         2'b01: pol   <=# 1 dat_i;        // POLARITY register         2'b10: mask  <=# 1 dat_i;        // MASK register       endcase      //     // generate dat_o     reg  dat_o;     always @(posedge clk_i)       case (adr_i) // synopsys full_case parallel_case         2'b00: dat_o <=# 1 { {{8-is}{1'b0}}, edgen};     //{  {{8-is}{1'b0}}, edgen}         2'b01: dat_o <=# 1 { {{8-is}{1'b0}}, pol};         2'b10: dat_o <=# 1 { {{8-is}{1'b0}}, mask};       endcase     //    // generate ack_o    reg ack_o;    always @(posedge clk_i)      ack_o <=# 1 wb_acc & !ack_o;  endmodule
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    12 years ago

    Did you achive to impelement multiple MSI interrupts using Cyclone IV GX?

    I am using IP_COMPILER_FOR_PCI_EXPRESS in Qsys but there is not a feature that enables multiple MSI interrupts. As i read in the manual, it is able to produce only one interrupt and the separation of the interrupts is able by reading the Interrupt Status Register.

    So, is it able to be done? If yes, what should I check?

    Thanks a lot.
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    12 years ago

    Hello,

    I'm not sure if anyone is still watching this post.

    I'm working on enabling mutiple MSIs for my PCIe card and it has been the most useful information I got so far!!

    When I try to manully apply the patch to my kenel, there's some incompatibility, like functions or variables not declared.

    What is the distribution you use to apply this patch on?

    Any information would be greatly appreciated!!

    Best Regards,
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    10 years ago

    Hi alterahenry,

    I am working on X86 PCIe Linux Driver to implement MSI-X Interrupt on Cycleon V FPGA board with reference design taken from Qsys.

    I have followed below steps in linux driver to enable MSI-X interrupt.

    1) Enabled PCIe Device

    2) Executed pci_enable_msix with total number of 4 vectors which is successfully enabled without any fail-case.

    3) After that, Disable Legacy Interrupt by setting 0 to 10th bit of PCI_COMMAND Configuration Space Register.

    4) Enabled DMA Mask Bit of PCI Cnfiguration (PCI_Command) Register.

    5) After that, Request IRQ based on getting vector value of MSIX Entries which allocated successfully without any issue.

    6) Than, I have triggered IRQ by writing data into IRQ source which writes some data into TXS Port but not getting interrupt event.

    Please find below sample code for more information.

    status = pci_enable_msix(dev, msix_entries, nr_entries);

    pci_read_config_dword(dev, PCI_COMMAND, &temp);

    pci_write_config_dword(dev, PCI_COMMAND, (temp & 0xFFFFFBFF)); /// To Disble Legacy Interrupt (set 0 to 10th bit)

    pci_read_config_dword(dev, PCI_COMMAND, &temp);

    /* Enable DMA Mask Bit of PCI Cnfiguration (PCI_Command) Register */

    if (dma_set_mask(&(dev->dev), DMA_BIT_MASK(32)))

    {

    dev_err(&dev->dev," No suitable DMA available for 32 Bit.\n");

    }

    else

    printk(KERN_INFO " Set 32 Bit DMA Mask Successfully...\n");

    if (dma_set_mask(&(dev->dev), DMA_BIT_MASK(64)))

    {

    dev_err(&dev->dev," No suitable DMA available.\n");

    }

    else

    printk(KERN_INFO " Set 64 Bit DMA Mask Successfully...\n");

    pci_set_master(dev);

    dev->irq = msix_entries[0].vector;

    if (request_irq(dev->irq, sls_sdhc_irq, 0, DRV_NAME, (void*)dev))

    printk(KERN_ERR "Error allocating interrupt.\n");

    else

    printk(KERN_ERR "Interrupt allocated successfully.\n");

    So, All above settings are configured successfully without any issue but I am no getting print from interrupt handler. so, it seems that interrupt is not generated yet.

    Please let me know if you have any solution or any idea for this type of issue.

    Regards,

    Ritesh Prajapati
  • Altera_Forum's avatar
    Altera_Forum
    Icon for Honored Contributor rankHonored Contributor
    8 years ago

    Hi,

    Can you please help me to understand how to enable multiple MSI Interrupts?

    In Qsys, if we select Number of MSI messages requested as 4, then will it support 4 different MSI interrupts?

    Also please share if anyone have Host software application to enable multiple MSI interrupts.

    Iam using Arria10 FPGA

    Regards

    Linus_alt