Agilex 7 E-Tile 100G Ethernet Bringup Issue

Hi,

I'm working with an Agilex 7 AGFB014R24A2E2V sitting on a development board with 2 x QSFP28 ports.

Now I'm trying to bringup a 100G Ethernet link between the FPGA and our IXIA test instrument. I'm using the E-Tile Hard IP for Ethernet and E-Tile CPRI PHY Intel® FPGA IPs. I have configured it for Single 100G without FEC and instatiated it in Platform Designer so that I have 2 instatiations, one for each 100G port and JTAG Avalon Master to get access to the Ethernet and Transceiver PHY toolkits and register readouts. I have a refclk of 156.25MHz and the TX PLL clock o_clk_pll_div64 is driving both i_clk_tx and i_clk_rx. So far there is no custom logic connecting to the Avalon streaming interfaces to transmit/receive any data on the MAC side.

So problem is a can't get the link up between FPGA and IXIA. I'm using QSFP28 100GBASE-SR4 modules to connect the ports with MMF-fiber.

FPGA => IXIA direction:

Ethernet Toolkit reports TX PCS Ready: up/green light and I can see (via SignalTap) that tx_lanes_stable is asserted. However, IXIA reports link down (red light).

IXIA => FPGA direction:

Ethernet Toolkit reports RX PCS aligned: down/red light and I can see (via SignalTap) that rx_am_lock, rx_block_lock, rx_pcs_ready are all deasserted.

I tried to run the PMA external adaption in Ethernet toolkit to optimize the PMA settings to see if that would get the RX PCS to get aligned. I also tried launching the Eye diagram viewer (in the Transceiver PHY Toolkit) but then I get the following error: eye viewer failed to show eye data: index 0 out of bounds for length 0.

Another thing I tried was to connect the 2 x FPGA ports together (using same QSFP28 100GBASE-SR4 modules). In this case the RX PCS aligns on both ports and rx_am_lock, rx_block_lock, rx_pcs_ready are all asserted.

So it seems connecting the 2 x FPGA ports works fine, but when moving one of them to our IXIA port link does not come up.

Do you suspect that my PMA settings need to be optimized to get the link up? PMA external adaption does not seem to do the job. Do you recommend using Eye diagram to do this? Do you know how to get my Eye diagram viewer to work?

One interesting observation I did while troubleshooting was when reading register 0x40080 in PMA Control and Status Register Map (found in E-Tile Transceiver PHY User Guide). For all 4 x transceivers I read 0x40080 = 0x 00030001 =>

tx_transfer_ready asserted

rx_transfer_ready asserted

tx_ready deasserted

rx_ready deasserted

rx_is_lockedtodata asserted

So this suggests that tx_ready/rx_ready from reset controller are not asserted. I reviewed my reset sequence that I perform for E-Tile Hard IP for Ethernet and E-Tile CPRI PHY Intel® FPGA IP, by asserting i_csr_rst_n, i_reconfig_reset. And after that also asserting i_rx_rst_n and i_tx_rst_n. I don't see any issue with reset sequence.

In table 57 in E-Tile Transceiver PHY User Guide I saw a comment that tx_ready/rx_ready might not get asserted if reset sequence not followed. But for me I'm doing the reset through the E-Tile Hard IP for Ethernet and E-Tile CPRI PHY Intel® FPGA IP so it should be covered by that reset?

Would you suspect that tx_ready/rx_ready issue might be related to my initial issue with link not coming up when connecting to IXIA?

Thanks in advance for you support!