Sorry for the delay. Here are more details..
The two Stratix-V parts on our board are each 5SGXEA7H2F35C3.
The board was designed symmetrically in the sense the same FPGA build file could be loaded into both FPGAs for full symmetric functionality.
Interlaken:
Between the two FPGAs there are four serial lines each operating an independent Interlaken link running at 12.4 Gbps.
Each FPGA receives the four Interlaken signals sent by the other. There is no channel bonding.
The GX transceivers running Interlaken are instantiated using the Native PHY IP core.
10G Ethernet:
Each Stratix-V part also uses some GX transceivers to interface SFP+ ports on the board.
On each FPGA two of those ports operate in 10G Ethernet mode.
The GX transceivers for those Ethernet ports are instantiated using a single 10GBASE-R PHY IP core configured for 2 channels.
For each Ethernet port an "Ethernet 10G MAC" IP core is also instantiated.
Interlaken troubles:
For the Interlaken links, on some builds, one of the four links reports a failure to achieve 67/64 PCS block lock.
The remaining three links so far operate correctly.
On a given build on some boards both directions of the link report the failure. On other boards there is no failure.
In rare cases we had a board on which one direction works while the other does not.
On a different build, all four links might work correctly.
For some builds we saw a strange phenomenon. The link reports block lock but when we pass certain payload in the Interlaken metaframes the CDR on the receive side fails to retain lock to data.
When the payload is removed the CDR lock succeeds again.Or with different payload the CDR locks fine.
When we pass the same payload that causes the trouble on the one link through a different Interlaken link the new CDR does not have trouble keeping lock.
Again, on other builds this may not occur.
10G Ethernet trouble:
Our design generates Ethernet frames for transmission and then forwards them through the Avalon interface into the MAC core which then forwards frames through the XGMII interface to the 10GBASE-R transceiver core.
We also support the reverse direction of receiving frames through the transceiver core and through the MAC core.
On some builds we pass frames into the MAC core as usual and the Avalon txstatus ports indicate transmission with no errors and yet the receive side of the Ethernet link does not indicate any frames. The Ethernet link is reported as up.
We sometimes see frames go out on one Ethernet port and not the other. The failing port could change from one build to another.
On some builds there are no transmit problems. On some boards and some builds we have receive problems in that the Ethernet frames are transmitted successfully and the Ethernet link is up but the Avalon receive port does not indicate frames.
New Contributor
