Calculate skew timing with Cyclone V and ksz9031

hello,

i have troubles with the ethernet bandwith and retransmissions (ack). after some diagnostic i think it is an timing issue.

the wire length from the fpga -> knz9031 is:

	mil	mm	m	delay	delay
RX_DV	1692,83	42,997882	4,30E-02	227,5E-12	228ps
RX_CLK	1649,35	41,89349	4,19E-02	221,7E-12	222ps
RXD0	1684,72	42,791888	4,28E-02	226,4E-12	226ps
RXD1	1648,24	41,865296	4,19E-02	221,5E-12	222ps
RXD2	1681	42,6974	4,27E-02	225,9E-12	226ps
RXD3	1677,12	42,598848	4,26E-02	225,4E-12	225ps

TX_EN	1625,87	41,297098	4,13E-02	218,5E-12	219ps
GTX_CLK	1629,75	41,39565	4,14E-02	219,0E-12	219ps
TXD0	1633,94	41,502076	4,15E-02	219,6E-12	220ps
TXD1	1657,65	42,10431	4,21E-02	222,8E-12	223ps
TXD2	1657,33	42,096182	4,21E-02	222,7E-12	223ps
TXD3	1655,12	42,040048	4,20E-02	222,4E-12	222ps

all wires are the same length, in every wire is an 22r resistor. the design is bought and not designed by us.

i found the following knowledge base entry:

https://www.intel.com/content/www/us/en/support/programmable/articles/000079123.html

=> TX_CLK to TXD/TX_CTL output data delay is specified as -0.85 to 0.15 ns

Datasheet 9031:
For the receive path (KSZ9031RNX to MAC), the KSZ9031RNX adds 1.2 ns typical delay to the RX_CLK output pin, with respect to RX_DV and RXD[3:0] output pins.

We use kernel v4 with u-boot and devicetree, after a lot of measuring and test,

i got the best result with this skew timing:

&gmac1 {
	status = "okay";
	phy-mode = "rgmii";
  /* 0 bis 900*/
	rxd0-skew-ps = <420>;
	rxd1-skew-ps = <420>;
	rxd2-skew-ps = <420>;
	rxd3-skew-ps = <420>;
	rxdv-skew-ps = <420>;

	txd0-skew-ps = <360>;
	txd1-skew-ps = <360>;
	txd2-skew-ps = <360>;
	txd3-skew-ps = <360>;
	txen-skew-ps = <360>;
/* 0 bis 1860*/
	rxc-skew-ps = <1680>;
	txc-skew-ps = <1740>;

	max-frame-size = <3800>;
};

But how could i calculate the skew timing?

Which timing should be the best, with mathematics

Thanks

35 Replies

Nurina
Regular Contributor
3 years ago
Hello,

Can you provide some updates?
I cannot get any confirmation with internal team on the best skew calculation.
But the normal practice is to start with measuring a working window, get the min and max skews then use the midpoint and fine tune it as you wish. I think you have done this already.

If it is working on the hardware then it should be fine.

Regards,
Nurina
- qwitza
  Occasional Contributor
  3 years ago
  hello,
  
  but how about the wire length?
  
  sould be the resistor considered?
  
  how would be the "best" timing for our board?
  
  thanks
Nurina
Regular Contributor
3 years ago
Hi,

We have a simulation board skew tool, this could help you calculate the best skew. https://www.intel.sg/content/www/xa/en/support/programmable/support-resources/design-guidance/board-developer.html

Regards,
Nurina
- qwitza
  Occasional Contributor
  3 years ago
  Hi,
  
  thanks again for your answer, but the skew parameter tool is only for the memory interface.
  
  there is no option for the cyclone v and no option for the rgmii interface.
  
  also it considered not the https://www.intel.com/content/www/us/en/support/programmable/articles/000079123.html from the cyclone v and the rgmii interface.
  
  i thought there is an intel employee who know how to calculate the desired skew values for the rgmii interface? all neccessary information at post #1.
  
  thanks again
Nurina
Regular Contributor
3 years ago
Hello,

Let me consult an RGMII expert on this.
I'll let you know of any updates.

Regards,
Nurina
Nurina
Regular Contributor
3 years ago
Hello,

Can you share your .qar file? You can also send it through e-mail if it is confidential.

Regards,
Nurina
- qwitza
  Occasional Contributor
  3 years ago
  hi,
  
  thanks again for your answer.
  
  i ve to wait until monday, because i need my collegue.
  
  we ve to send it via e-mail, where can i found your e-mail address?
  
  thanks
Nurina
Regular Contributor
3 years ago
Hello,

I have sent you an email. Please search for an email with subject "RE: Intel Customer Support - Case #: 05709344"

Thanks,
Nurina
Nurina
Regular Contributor
3 years ago
Hello,

Thank you for sending your .qar file, I have sent this to internal team and they will give suggestions soon.

Regards,
Nurina
Nurina
Regular Contributor
3 years ago
Hello,

The wire lengths in your PCB are all the same right?
Is there resistors in every wire? Where do you have this resistor?

Regards,
Nurina
- qwitza
  Occasional Contributor
  3 years ago
  Hello,
  
  the wire length is in my first post, and yes they have all very near the same length.
  
  the ksz9031 and the cyclone v are on the top layer, the resistors are on the bottom layer. the resistors are placed near the ksz9031.
  
  thanks for your work, and answer
Nurina
Regular Contributor
3 years ago
Hello,

I am had problems trying to add comment on here so I have sent you a response through email.
Please let me know if it helps.

Regards,
Nurina
- qwitza
  Occasional Contributor
  3 years ago
  Hello,
  
  i ve got only one e-mail with same message above, nothing else.
  
  i ve no problems to comment here.
  
  greetings
  - qwitza
    Occasional Contributor
    3 years ago
    Hello,
    
    the e-mail was in the spam folder.
    
    thanks
Nurina
Regular Contributor
3 years ago
Hello,

Can you provide some updates?

Regards,
Nurina
Nurina
Regular Contributor
3 years ago
Posting the solution here for the community's future reference.

Regarding the PCB traces, resistors etc.

Typically we take into account the track lengths on PCB, line impedance and capacitive loading. All of which are inter-related. The track delay data you have is quite impressive and has pretty high tolerance to variations. Therefore, track delay can be ignored from analysis.

TX Timing

Examining the RGMII 2.0 spec: (see attached)

At the transmitter, TX_CTL & TXD are sent edge-aligned to TXC by the transmitter. However at the receiving end, the TXC signal is center-aligned to data. According to the spec, the skew can be achieved by PCB trace routing or by an internal delay in the transmitting or receiving node.

Assuming the following:

With the HPS we can’t add 2.0 ns of delay to TXC

The customer hasn’t added the skew via PCB

All tracks on PCB are delay matched

Paths are delay matched within the PHY package

Then the only option is to add the delay within the PHY.

Here’s what we publish wrt TX:

The PHY allows for internal delays on TX_CLK via MMD register 2.8. The goal would be to center-align the signals to TX_CLK by delaying TX_CLK.

As a default, the PHY adds a 15*0.06ns = 0.900 ns delay to TX_CLK. However to center-align, TX_CLK should be delayed 1.65 ns.

Therefore if the customer sets MMD register 2.8[9:5] to int(1.65/0.06) = 27 or 1.62ns. This will provide a Tsu of 1.47 ns and Th of 1.53 ns. This should easily meet the TsetupR & TholdR requirements of 1.0 ns.

RX Timing

Examining the RGMII 2.0 spec:

Like the Tx timing, the PHY sends its data edge aligned to RXC. At the receiving end, the RXC signal is center-aligned to data. According to the spec, the skew can be achieved by PCB trace routing or by an internal delay in the transmitting or receiving node.

Assuming the following:

With the HPS we can’t add 2.0 ns of delay to RXC

The customer hasn’t added the skew via PCB

All tracks on PCB are delay matched

Paths are delay matched within the PHY package

Then the only option is to add the delay within the PHY.

Here’s what we publish wrt RX:

As before, the PHY allows for internal delays on RX_CLK via MMD register 2.8. The goal would be to center-align the signals to RX_CLK by delaying RX_CLK.

Examining Table 4-6 from above, as a default, the PHY adds a 15*0.06ns = 0.900 ns delay to RX_CLK. However to center-align, RX_CLK should be delayed 2.00 ns. Therefore if the customer sets MMD register 2.8[4:0] to int(2.0/0.06) = 33 or 1.98ns. This will provide a Tsu of 1.98 ns and Th of 2.02 ns. This should easily meet the TsetupR & TholdR requirements of 1.0 ns.

Final Notes

The above analysis is fairly simplistic. It doesn’t take into account duty-cycle distortion, jitter, IO skew, PCB skew, or variations over PVT. Here’s a TI appnote the tries to address some of these concerns.

https://www.ti.com/lit/an/snla243/snla243.pdf

Lastly, the customer can scope these signals & validate the clock-data relationships.