Forum Discussion

xytech's avatar
xytech
Icon for Contributor rankContributor
7 years ago
Solved

Arial10 Power-Up Calibration Clock Question

Hi, we are using ​10AX057H3F34E2SG for PCIe GEN3.0 x4 lanes, connected to a PC module by Board to Board connects.

I read on section 7.3 of UG-01143 (Arria 10 Transceiver PHY User Guide, page582) that user need to provide a CLOCK for proper transceiver for power-up calibration, and only Power-ON Calibration are allowed for PCIe..

​

Question1: Is this CLK refer to CLKUSR or dedicated XCVR REFCLK pin pairs, or both? As marked with red and blue lines in below picture. I guess both CLKUSR and XCVR clock should be provided to FPGA. But still need intel confirm.Question2: For section 7.3, what's the specific definition of "Programm" action in first sentence? Does it refer to Configuration, or Initialization, or both? Also, I'm confused that seems to be conflict between the FIRST and SECOND sentence for this paragraph. First sentence indicates that XCVR calibration start AFTER "programmed" , while second sentence indicates that calibration could continue DURING "Programming

", and may extend into user-mode.

How to understand this? I guess the seuqence should be Power on -->Configuration-->Initialization-->XCVR Calinration( user mode in parallel). Not sure......When EXACTLY the calibration begins? Is there timing diagrams on Intel docs?

Question3: To avoid the well-known “100 ms boot time requirement for PCIe”, We decide to power FPGA first, after detect the CONFIG_DONE signal be pulled high, then start PC, then the PCIe reference clock will be sent from PC to FPGA GXB banks.

However, this will be CONFLICT with UG-01143 section 3.2 , page381 note, which writes “To successfully complete the calibration process, the reference clocks driving the PLLs (ATX PLL, fPLL, CDR/CMU PLL) must be stable and free running at start of FPGA configuration”. Do you have any insight about how could we solve this conflict?

For now, we use ASx4 config scheme, and no likely to change it for other config schemes.

Thanks very much!

​

​

​

​

  • JonWay_altera's avatar
    JonWay_altera
    7 years ago

    Hi @xytech​

    Calibration starts during configuration.

    No timing sequence, the time for calibration to complete vary by device.

    Based on experience, faster configuration time is needed if you are designing for open-system to meet the 100ms. For closed-system, you can adjust the PERST to allow more time.

    Hope this helps.

17 Replies

Show More
  • JonWay_altera's avatar
    JonWay_altera
    Icon for Frequent Contributor rankFrequent Contributor
    7 years ago

    Hi @xytech​

    Calibration starts during configuration.

    No timing sequence, the time for calibration to complete vary by device.

    Based on experience, faster configuration time is needed if you are designing for open-system to meet the 100ms. For closed-system, you can adjust the PERST to allow more time.

    Hope this helps.

    • xytech's avatar
      xytech
      Icon for Contributor rankContributor
      7 years ago

      Hi JwChin, thank you very much for patient reply.

      ​

      As UG-01143 section 7.3 writes, PCIe links do not allow user recalibration. We have no choice. Only can do power up calibration.

      OK, finally confirmed that Calibration starts during configuration and no specific timing, cal_busy indicates Calibration completes.

      ​

      Yes, we do design OPEN-System. Root PCIe node is a standard x86 COMe module (compact PC module), and we cannot control when PERST# will be released.

      ​

      Actually in our older product, we use Artix-7A200T FPGA of Xilinx, which DO NOT explicitly requires “transceiver Reference clock must be stable and free running BEFORE FPGA powered up for successful XCVR calib”(they do not need CLKUSR neither) , so our system are designed to power FPGA first, after detect its CONFIG_DONE signal released (pulled high), then we start PC module, this made sure PC can enumerate FPGA PCIe node. It worked fine.

      ​

      Now it’s clear that this solution is NOT suitable for Intel A10. So we must change a fast enough configuration scheme to satisfy PCIe 100ms requirement. Hardware and/or Logic design need modifications inevitablely.

      ​

      Thanks again for your kind help. Good day

  • JCann's avatar
    JCann
    Icon for New Contributor rankNew Contributor
    7 years ago

    With an Arria 10 design at that density, I would recommend looking at CvP (configuration via protocol). My understanding is that this is meant to load the periphery of FPGA (including PCIe core) in order to satisfy the 100ms enumeration requirement. Then the "core" FPGA (rest of the logic) is loaded after the PCIe interface is safely up. Looks like datasheet calls out minimum config time for that device of >600ms when using AS (active serial) x4. Best case for FPP is slightly less than 100ms.

    Regarding calibration, both clocks should be required (PCI_REFCLK and CLKUSR). CLKUSR toggles PreSICE state machine that runs the calibrations and you need the REFCLK for the PLLs. Keep in mind that in A10 devices, PCIe (w/ hard IP) are the first devices that calibrate and then cascade the CAL* status bits downstream to other GXBs on the chip. If you have a PCIe interface instantiated in your project that's not hooked up to anything (so not successfully calibrating), it will gate other GXBs from calibrating. Running into that right now with a design.

    • xytech's avatar
      xytech
      Icon for Contributor rankContributor
      7 years ago

      Hi JCann,

      Thanks for reply. We use 2 GXB banks for PCIe GEN3.0 x8 linked to a Standard PC module. The rest 2 GXB banks are used for communication between two Arial 10 FPGAs.

      Just to confirm I percept your second paragraph correctly, is following what you mean?

      ------After FPGA is powered up, among all GXBs, PCIe hard IP GXBs are firstly calibrated, and then certain CAL* status bits/signals are cascaded (delivered) downstream to other 2 GXBs for calibrating. If PCIe GXBs is not successfully calibrated, it will GATE(Prevent) other GXBs from calibrating, which means finally all GXB banks will not be calibrated successfully , and cannot function correctly. ------

      Furthermore, as UG-01143 writes, “Transceivers include both analog and digital blocks that require calibration to compensate for process, voltage, and temperature (PVT) variations”

      Just wondering, if it’s not calibrated successfully, how will transceivers behave? Will it become unable to work completely, or work with de-rated performance, such as lower speed rate, worse BER?

      And if we enable datas compression funtion in Quartus to reduce bits stream file's size(typically reduce by 30%-55%), will the config time be reduced?

      Thanks for helping!

  • JCann's avatar
    JCann
    Icon for New Contributor rankNew Contributor
    7 years ago

    Given your specific implementation - my understanding of how A10 GXBs work (with PCIe) is that both of the PCIe banks need to be connected to a host and properly calibrated in order for the 2x GXBs that are used for inter-FPGA communication can be properly calibrated. This is assuming that both PCIe Gen3x8 are using the Hard IP (HIP). Rules might be different if not using HIP, but I would guess that you went the route of HIP for this device. The use case that I have is using another piece of DisplayPort IP that has calibration logic built into it. I can't speak to whether you would expect de-rated performance on a link where you have control over PLL+reconfig+PHY and could possibly "trick" it into thinking that it's OK based on lack of calibration (not recommended).

    I just wanted to point out that there are some interesting dependencies that need to be accounted for when using PCIe HIP that might or might not be connected.