PCIe Enumeration Failure for CXL IP
When attempting to validate the Agilex 7 R-Tile Compute Express Link (CXL) 1.1/2.0 IP (Type 2 and Type 3) using a CXL compatible host server, the host server is unable to complete PCIe bus enumeration. The host server stalls while attempting to complete PCIe bus enumeration. The stall never resolves after boot, and access to to the host is never granted. Depiction of the stall and its status code from the host server's perspective is provided as an attached PNG file titled: "pcie_enumeration_stall". Debugging Information: A PCIe Gen 5.0 reference design using the Altera R-Tile Avalon Streaming IP For PCI Express was used to validate that PCIe enumeration could complete fully without failure, and that the host server could exchange data with the FPGA. While running the CXL example design, the Quartus System Console's Link Logger indicates that the LTSSM state is in the "UP_L0" before the PCIe bus enumeration stall. The state may sometimes change when attempting to "Refresh" the status during the PCIe bus enumeration stall. The state may briefly enter recovery (UP_L0 -> REC_IDLE -> REC_RCVRCFG -> REC_RCSVLOCK -> REC_COMPLETE -> UP_L0). Depiction of the Quartus System Console's Link Logger when this occurs is provided as an attached PNG file titled: "ltssm_link_logger". While running the CXL example design, the Quartus System Console's Link Logger indicates that the advertised and negotiated link speeds and widths are both 32.0 GT and x16. Depiction of a CXL Type 3 Quartus System Console's Overview is provided as an attached PNG file titled: "cxl_ip_systemconsole_overview". Instead of generating the example design, the pre-compiled binary offered by Altera for Type 2 and Type 3 CXL IP designs was used and resulted in the exact same failures as shown above. CXL.mem transaction registers (M2S and S2M) are 0x00, indicating that the host server never progresses far enough to begin sending/receiving transactions/requests. Between the PCIe build that functions and the CXL build that does not function (stalls at enumeration), no server UEFI settings were changed. A CXL enable function was enabled for all tests. Several PCIe UEFI settings were changed in an attempt to resolve the enumeration stall, but none changed the outcome. Attempting to disable the CXL Compliance 2.0 and the HDM decoder registers both did not resolve the issue. The FPGA was powered and programmed before the server was launched. Two different CXL servers were tested and both resulted in the same behavior. The relevant PCIe and CXL settings from BIOS is provided as an attached PNG file titled: "cxl_server_settings". The CXL REFCLK was tested as both COMMON and SRIS/SRNS. Each test changed SW3 to use relevant onboard and connected based clocks. IP Settings: CXL IP settings are uploaded as PNG files titled: "cxl_ip_settings_N". The settings tested are the default provided settings as well as a version with a 300 MHz PLD clock (SRIS). Hardware Details: FPGA is connected to host server via PCIe Gen 5.0 x16 slot on Tile 14C. FPGA device is the Altera Agilex 7 FPGA I-Series Development Kit (Production 2x R-Tile & 1x F-Tile) (AGIB027R29A1E1VB) The DIMM provided with the development kit is slotted into DIMM Slot A. SW1 is set to 1000 (PCIe PRSNT x16). SW3 is set to 0110 for designs using the CXL/PCIe common clock and 0000 for designs using the CXL/PCIe onboard REFCLK (SRIS). Software Details: Quartus Prime Pro Edition v25.1 was used to generate the designs. R-Tile Altera FPGA IP for Compute Express Link (CXL) was generated with version 1.17.0. FPGA Design: The FPGA design is generated using the example design with the IP settings given above. A pre-compiled binary provided by Altera was also used to test instead of a generated design. Server details: SMC AS-1126HS-TN (CXL 2.0 via 4x PCIe gen5 x16 slots) CPU: 2x AMD EPYC 9135 (CXL 2.0) RAM: 4x Micron 64GB @ 6000 MT/s UEFI: AMI 1.7a 10/30/2025 Attachments: The system console debug register outputs are saved to CSV files attached to this post. These CSV files are taken from a CXL Type 3 reference design with PLD REFCLK at 300 MHz (SRIS). Questions: Can you provide guidance on how to obtain more information on the enumeration status other than the LTSSM register? Can you provide the UEFI/BIOS settings for PCIe/CXL that was used to test this IP as reference? Could the configuration space registers (DVSEC/HDM) or the TLP handling implemented in the CXL example design RTL create this PCIe enumeration failure? Can you provide guidance on what debug/status registers the CXL IP provides that could be relevant to this issue?
Reconfigure Agilex-7 with PCI IP causes system reboot
Hi We have a PCI with Alteras Agilex-7 eval board connected to a Intel Sierra Forest server running Ubunto 24, when reconfigure our Agilex-7 eval board the PC reboots often which causes alot of frustation since we are not alone on the PC server. what is the proper procedure when needing to configure with a PCI IP, how to issue a re-enumeration? What is the procedure when needing to reconfigure a design that has a PCI IP ThanksCyclone 10 GX Dev Kit: IOPLL not locking when using CvP mode
Hi everyone, I'm working with a simple CvP-based design on the Cyclone 10 GX Development Kit. My design is based on the official CvP example design, with a small modification: I've added an IOPLL that generates 150 MHz and 125 MHz clocks from a 50MHz input clock. Here's the issue I'm seeing: After power-up, I program the *.core.rbf file using quartus_cvp.exe, and everything works as expected – the IOPLL locks reliably. However, if I program the exact same *.core.rbf file again (without powering off), the IOPLL no longer locks. Has anyone encountered similar behavior or have any idea what might be causing this? Is there something specific I need to reset or reinitialize in the PLL or CvP flow when reprogramming? For clarification I've added my top-level HDL design. Thanks in advance for any help or insights!
Behavior of 10 GX Avalon-MM Interface for PCI Express* IP Core when byteenable=16'h0000
Hi, we are using an Arria® 10 and Cyclone® 10 GX Avalon® Memory-Mapped (Avalon-MM) Interface for PCI Express* IP Core. During our tests we noticed some illegal PCIe packages generated presumable due to a wrong data length. We could tackle down the problem to the following basic setup: avalon_mm_master => 128 bit bus => PCIe-Core When we send the following sequence (two words), we get an illegal/unexpected PCIe transfers/behavior: burstcount = 2, address = address_a, data = some_data, byteenable=16'h0000 burstcount = 1, address = address_a+16, data=some_data, byteenable=16'hFFFF When we only send the second word everything works fine. This sequence originally comes from a qsys autogenerated 256=>128 width change in the interconnect somewhere upstream in our project. My question is: Do we miss something here? Does the IP-Core not allow for a first word to be completely disabled? If so, is there any (automatic) way to tell qsys / the interconnect to discard a leading all_bytes_disabled word? 5.3. 64- or 128-Bit Bursting TX Avalon-MM Slave Signals Best regards, MichaelError: dut.p0_hip_status has no associated reset.
Hello Altera I am using Quartus Prime Pro 25.1.0 build 129 03/26/2025 SC Pro Edition. In my Agiliex 7 Design project, I have Intel R-Tile MCDMA for PCI Express intel_pcie_rtile_mcdma Version 5.3.1 Ip instantiated as "dut". in parameters settings I have enabled to have hip status interface. https://www.intel.com/content/www/us/en/docs/programmable/683821/25-1-1/hard-ip-status-interface.html this interface is connected to a custom design QCP file. The QCP uses "app_clk" and "app_nreset_status" from dut ip as its clk and reset inputs. They go through clock bridge and reset bridge. Clock and reset outputs from these bridges are used internally in custom logic code. In platform designer as I connect "dut.p0_hip_status" and "custom_module.pcie_ep_hip_status_in" ports, I get an error as following "Error: pcie_ed: Interfaces custom_module.pcie_ep_hip_status_in and dut.p0_hip_status must have matching associated resets, but dut.p0_hip_status has no associated reset." This Error does not make sense to me, as dut`s hip status interface and my qcp`s status interface ports shows correct clock and reset association in component instantiation tab; and my custom design uses same clock and reset to its clock and reset bridge inputs. can you please help me to understand what is this error about and how do i resolve it?
Agilex 5 GTS MCDMA at AXI-MM mode not working
I am using Quartus 25.1 and wanted to test the new GTS MCDMA for Agilex 5 in AXI-MM mode. I generated the example design, which uses AXI-ST mode, and tested it with the provided driver. Everything worked fine—the device was identified, and the driver functioned correctly with the user application called "perfq_app". According to the user guide, this is referred to as the "Custom Driver." The next step I took was to switch from AXI-ST to AXI-MM, but now the driver doesn't work. I compared the provided code with the example design code for Agilex 7 MCDMA. The only differences I found were two macros: one for the data width and the other to indicate which example design it is. I generated the AXI MCDMA example design at memory mapped mode for the Agilex 7 and use the provided code and it worked fine. (On the left is the Agilex 5 provided code, and on the right is the Agilex 7 version.) I tried changing IFC_QDMA_ST_LOOPBACK to IFC_QDMA_MM_LOOPBACK, but it didn’t help. I even connected Signal Tap to the DMA MM master interface and saw that no transactions were occurring on this interface. Has anyone encountered this issue or have any idea how to solve it?Implementing MSI-X in Agilex 7 Rtile MCDAM PCIe based design
I am using Agilex 7 - AGIB027R29A1E2VC FPGA. On Rtile I am using Multi Channel DMA FPGA IP for PCI Express - Data Movers Only. https://www.intel.com/content/www/us/en/docs/programmable/683821/25-1-1/data-mover-only.html My design currently performs H2D and D2H DMAs as I need to. Now I want to send MSI-X interrupt after certain DMA completion and thats where I need help. Specifically "MSI-X Enable" and "MSI-X function mask" related functionality. Other Parts related to implementing MSI-X is clear to me. "MSI-X Enable" and "MSI-X function mask" both can be written by host side driver and the interrupt controller inside PCIe ep device is supposed behave accordingly. Currently I am struggling to figure out how can I get a copy of CFG writes to MSI-X capability structure when CFG WRs are targeting "MSI-X Enable" and "MSI-X function mask"? If I get a copy of such CFG writes than my application layer logic can keep track of latest value written and update MSI-X behavior as intended. https://www.intel.com/content/www/us/en/docs/programmable/683501/25-1-1/msi-x.html mentions that "The R-Tile IP for PCIe provides a Configuration Intercept Interface. User soft logic can monitor this interface to get MSI-X Enable and MSI-X function mask related information." https://www.intel.com/content/www/us/en/docs/programmable/683501/25-1-1/configuration-intercept-interface.html mentions that "The Configuration Intercept Interface (CII) allows the application logic to detect the occurrence of a Configuration (CFG) request on the link and to modify its behavior." is there any example on how CII interface can be used to monitor MSI-X capability structure accesses? https://community.intel.com/t5/FPGA-Wiki/Implementing-MSI-X-for-PCI-Express-in-Altera-FPGA-Devices/ta-p/735678 this link appears to be broken furthermore, table 71 on https://www.intel.com/content/www/us/en/docs/programmable/683501/25-1-1/configuration-intercept-interface.html provides cii interface signal list and its details - after reviewing them I have following questions. pX_cii_wr_o Output Indicates that cii_dout_p0/1 is valid. This signal is asserted only for a configuration write request. there is no mention of "pX_cii_rd_o" signal, can pX_cii_wr_o signal be used to determine read request? if this signal is at logic 0 , does it mean that this is CFG read request? pX_cii_addr_o[9:0] Output The double-word register address in the received TLP header on the CII. is this the address associated with CFW write? how do i know Config space offset at which MSI-X capability is located? So I can compare pX_cii_addr_o along with pX_cii_hdr_first_be_o to check if the CFG access to targeting "MSI-X Enable" or"MSI-X function mask" or not. how do I use CII interface so it ignores all offsets other than MSI-X capability structure relatedR-Tile Avalon® Streaming Intel® FPGA IP for PCI Express Testbench
Hello, our research team is working on PCIe controller with Agilex-7 M Series. We are using R-Tile Avalon® Streaming Intel® FPGA IP for PCI Express. According to Using the Avery BFM for R-Tile PCI Express Gen5 Simulations in R-Tile Avalon® Streaming Intel® FPGA IP for PCI Express User Guide, (https://www.intel.com/content/www/us/en/docs/programmable/683501/25-1-1/overview-59246.html) Testbench files for Avery BFM is supported. (Figure below, link above) Could you provide us testbench sample for Avery simulation?
How to implement PCIE hard IP for Cyclone IV GX
Hi, I am currently working with a Cyclone IV GX FPGA: EP4CGX75CF23C7. I would like to run a PCI transceiver and map the memory (on-chip memory) to the endpoint using 1 data lane. What do I need for this in quartus II 23.1 or 24.1? I see IP_Compiler for PCI Express but I cannot find any compatible instructions on how to use it. Thanks.
