Forum Discussion

Sajeev_antony's avatar
Sajeev_antony
Icon for Occasional Contributor rankOccasional Contributor
5 hours ago

Stratix 10 E-tile Hard IP for Ethernet, 10GE with external AIB clocking error

**Product:** Stratix 10 TX (E-tile), Quartus Prime Pro 26.1.0 Build 110. IP: E-Tile Hard IP for Ethernet, core variant "100GE or 1 to 4 10GE/25GE with optional RSFEC and 1588 PTP", 1 channel, active 25GE, MAC + PCS, RS-FEC off, PTP off, AN/LT off, external AIB clocking on, "Enable custom rate" on (also seen without it).

## Set-up

- External AIB clocking as in the 10G/25G dynamic reconfiguration design example: E-tile Native PHY in PLL mode (refclk 156.25 MHz) gives i_aib_2x_clk = 805.66 MHz and i_aib_clk = 402.83 MHz; i_sl_clk_tx / i_sl_clk_rx = i_aib_clk at both rates.

- 25G <-> 10G switching with the sequence of the example's c3_reconfig.c (TX/RX ratio attributes 0x0005/0x0006 = 0xA5 / 0x42, Ethernet registers 0x40B / 0x37A / 0x37B, width 0x0014 = 0x55, phase slip 0x000E, PMA enable, RX initial adaptation high effort).

- Client: Avalon-ST, ready latency 0, 64-byte frames, about 9.5 M frames/s per direction at 10G. Two channels on the same tile linked through SFP28 modules and fibre.

## Results

- **25GE: no errors** (tens of millions of frames, MAC FCSERR 0; timed replay of 17,562 frames bit-exact).

- **10GE, RX FCS errors (MAC RX statistics):**

| Configuration at 10G | One direction | Full duplex | Internal serial loopback |

|---|---|---|---|

| TXMAC_EHIP_CFG flowreg_rate = 4, i_sl_custom_cadence = 0 (design-example setting) | 5.1 % | 0.7 / 0.8 % | -- |

| flowreg_rate = 7, i_sl_custom_cadence = 64 of 165 cycles (156.25 / 402.83 MHz) | 4.0-4.2 % | 0.5-1.6 % | 0.74 % |

| flowreg_rate = 7, cadence 2 of 5 (161.13 / 402.83 MHz) | 10 %, 3.2 % of frames lost | 1.3-8 % | -- |

| flowreg_rate = 7, cadence 60 or 62 of 165 | -- | 2-3 %, RX counts more frames than sent | -- |

 

- The error rate is repeatable (4.0-4.2 % over six TX datapath resets) and lower when the receiving channel is also transmitting.

- TX FIFO overflow / underflow (o_sl_txfifo_overflow / underflow) are seen on the transmitting channel. Using o_sl_txfifo_pfull / pempty to modulate the cadence made no difference.

- **PMA PRBS31 at 10.3125 Gbps (hard PRBS generator / checker, attributes 0x0002 / 0x0003 / 0x0017 / 0x0018 / 0x001A): 0 errors** in internal serial loopback (20 s) and over the fibre in both directions (30 s, > 3e11 bits each). The checker reads 0xFFFFFFFF with non-PRBS input.

 

## Questions

1. Is 10GE with external AIB clocking and a 402.83 MHz client clock supported without frame errors? What TXMAC_EHIP_CFG flowreg_rate value and i_sl_custom_cadence pattern are required?

2. Are other EHIP, PCS or AIB registers (TX / RX FIFO thresholds, AIB adapter settings) required at 10G with external AIB clocking, beyond those in c3_reconfig.c?

3. If not supported, what is the recommended clocking for run-time 10G <-> 25G switching with the Ethernet hard IP? Our earlier attempt with o_clk_pll_div64 driving i_sl_clk_tx / rx stopped the Avalon-MM register interfaces while the line rate was changed.

2 Replies

  • ai_responder's avatar
    ai_responder
    Icon for Community Manager rankCommunity Manager

    Thank you for your inquiry. To ensure an accurate response, your request has been routed to an Altera application engineer for further review and assistance.


    Note: To continue using AI Responder, please start your next inquiry with ai_responder

    Disclaimer (Beta): The AI Responder is currently provided as a beta feature. AI-generated responses may contain errors or omissions. Please verify technical guidance with official Altera documentation before implementation.

    Thank you very much.

  • Sajeev_antony's avatar
    Sajeev_antony
    Icon for Occasional Contributor rankOccasional Contributor

    ## 1. What we did since the first report

     

    **Workaround in use (works):** each channel has its own glitch-free clock switch (Stratix 10 Clock Control IP) in

    front of i_aib_clk / i_aib_2x_clk and i_sl_clk_tx / i_sl_clk_rx:

    - 25G: 402.83 / 805.66 MHz from an E-tile Native PHY in PLL mode (as in the design example).

    - 10G: 161.13 / 322.27 MHz from a second Native PHY PLL (refclk 156.25 MHz), TXMAC_EHIP_CFG flowreg_rate = 3.

    - Switch sequence: PMA disable, analog reset, TX / RX ratio, clock switch while the channel is held in reset, channel

    reset, Ethernet registers 0x37A / 0x37B, width, phase slip, PMA enable, RX adaptation (from c3_reconfig.c).

     

    Results with the workaround: 10G and 25G, any channel at any rate independently, 0 FCS errors in all tests (80+

    traffic tests of 2 s, 5-round switching loops, timed replay of 17,556 frames bit exact, PMA PRBS31 600 s with 0 errors

    at both rates). 25G + RS-FEC (Clause 108) on single-channel instances also works, including run-time switching

    25G <-> 25G + RS-FEC <-> 10G with the register values of c3_reconfig.c.

     

    ## 2. Re-test of the documented scheme (402.83 MHz AIB clock at 10G), 30-Oct-2026

     

    Same image, two channels on one E-tile (ch 16 and ch 1) linked by SFP28 modules and fibre. 10G, TXMAC_EHIP_CFG =

    (read & 0x7E07) | 0x9FFF8108 (flowreg_rate 4, as c3_reconfig.c 25gnofec_to_10g and as the register description "4: 10G

    (with PTP or when External AIB clocking is enabled)"), XUS_TIMER_WINDOW 0xC4E3, BER_INVALID_COUNT 0x10, i_aib_clk /

    i_sl_clk = 402.83 MHz. MAC statistics, 2 s per test:

     

    | Test | Result |

    |---|---|

    | Reference: both channels 10G on 161.13 MHz, flowreg 3 | 14.21 M frames each way, 0 FCS |

    | flowreg 3, both channels on 402.83 MHz | TX not throttled (30.2 M frames accepted), TX FIFO over / underflow, far end 1.0 M good + 8.9 M FCS errors |

    | flowreg 4, both channels on 402.83 MHz, full duplex | MAC throttles to line rate (19.4-19.8 M frames), no TX FIFO over / underflow, 0.84 % FCS each way; RX status shows hi-BER / fault / CRC events |

    | flowreg 4, only channel A on 402.83 MHz, B on 161.13 MHz | B -> A (RX on 402.83 MHz): 14.21 M, **0 FCS**; A -> B (TX on 402.83 MHz): **2.4 % FCS** |

    | flowreg 4, A transmitting alone (simplex) A -> B | run 1: 1 FCS error in 18.7 M; run 2: 9.3 % FCS |

    | flowreg 4, timed replay of a real 10G traffic pattern (17,556 frames, below line rate), A -> B, 5 runs | every run 5.4 % FCS (936-947 frames) and 3,054 frames sent late (the MAC accepts data slower than the schedule needs); same replay on 161.13 MHz: 0 FCS, 0 late |

     

    Conclusion: with a 402.83 MHz AIB / client clock at 10G the **receive** path is error free, the **transmit** path

    corrupts 2-9 % of the frames (varying between TX restarts at line rate, constant 5.4 % with real traffic) and cannot

    keep a timed schedule. This matches the 27-Oct report (custom cadence variants also failed).

     

    ## 3. Why the documented scheme matters for us now: RS-FEC on more ports

     

    - One RS-FEC block serves 4 lanes, and one IP instance must own the whole block (E-Tile PHY UG 683723 p48; our Fitter

    error 175001 with two single-channel instances on ch 0 and ch 1). To have RS-FEC on several ports of one block we

    need a multi-channel instance.

    - The multi-channel instance (generated with 4 channels, first_lane0) has **one** i_aib_clk / i_aib_2x_clk for all

    channels, so our per-channel 10G clock is not possible inside it. Per-channel 10G / 25G then requires the

    402.83 MHz scheme of Ethernet UG 683468 p172-173 (Figure 63: "You can reconfigure the master channel to any mode

    without bringing down the slave channel functionality"), which fails on TX as shown above.

     

    ## 4. Questions

     

    1. Is 10GE TX with external AIB clocking at 402.83 MHz and flowreg_rate 4 expected to work without frame errors? If

    yes, which further settings are needed (EHIP / PCS / AIB adapter registers, TX FIFO thresholds, i_sl_custom_cadence,

    ready latency)? Can you provide a configuration that has been verified in hardware with continuous traffic at line

    rate?

    2. In a multi-channel 10GE/25GE instance with external AIB clocking, is there any way to clock individual channels

    differently (e.g. per-channel AIB clock), so that one channel runs 10G at 161.13 MHz while another runs 25G?

    3. Is our per-channel clock switch (Clock Control IP in front of i_aib_clk / i_aib_2x_clk / i_sl_clk, switched only

    while the channel is held in reset) a supported use of external AIB clocking?

    4. PHY UG 683723 p117 Figure 60, note 1 on RSFEC_1 (ch 4-7) and RSFEC_4 (ch 16-19): "This block cannot be used in

    combination with EHIP_CORE - fractured bypass." Please explain the restriction: we use RSFEC_4 (ch 16, first_lane0)

    with a single-channel 25G + RS-FEC instance (Fitter accepts it) and plan RSFEC_1 (ch 4).

    5. Documentation remark: in the DR example c3_reconfig.c (25gnofec_to_10g) the comment says "flowreg_rate 3 -> 4

    3:10G, 4:25G", while the register description (Ethernet UG 683468 p228) says "3: 25G or 10G (without PTP), 4: 10G

    (with PTP or when External AIB clocking is enabled)". The code writes 4 for 10G; the comment appears to be reversed.