Forum Discussion

Daehyeok_Kim's avatar
Daehyeok_Kim
Icon for New Contributor rankNew Contributor
5 years ago
Solved

Link status on BD-NVV-N3000-3 card

Hello, I've installed a BD-NVV-N3000-3 card on our server and connected it to an Intel Tofino-based 32-port 100Gbps switch using a QSFP+ cable. And I've followed the instruction in the user guide (...
  • JonWay_altera's avatar
    JonWay_altera
    5 years ago

    Hi @Daehyeok_Kim

    Thanks for your update on your debug status. Im glad to know that we have ruled out the cable or card issue. For the switch related queries, I am not the subject matter expert. Could you file a separate forum thread on the Ethernet products Forum (https://community.intel.com/t5/Ethernet-Products/bd-p/ethernet-products). They will be able to guide you from there. Appreciate if you can put in the link to the new thread for the benefit of others who face similar issue.

JonWay_altera's avatar
JonWay_altera
Icon for Frequent Contributor rankFrequent Contributor
5 years ago

1) Thanks. I think you have answered me.
OPN refers to BD-NVV-N3000-3
Version loaded refers to the MAX10 build/NIOS version + Bitstream ID.

2) Yes, loopback module as shown in the link you referred: https://www.intel.com/content/www/us/en/programmable/documentation/zsf1588015530773.html#xzd1588094394268

No, it is not included in the package. You will have to purchase it separately from the module makers.

If you do not have a loopback module at hand, you can order for 2 (one for each cage). It will be useful for future debugging as well. Meanwhile, you can try connect the ethernet cable from QSFP cage A to cage B, and check fpgainfo phy. If port is up, then it would be the connection to the external switch issue. If it is still down, it would either be a card or a cable issue. To rule out if it is a cable issue, you will definitely need a loopback module.

Daehyeok_Kim's avatar
Daehyeok_Kim
Icon for New Contributor rankNew Contributor
5 years ago

Thanks! We just checked the port status by connecting two cages using an Ethernet cable, but
"fpgainfo phy" command still shows they are down:

Board Management Controller, MAX10 NIOS FW version D.2.1.24
Board Management Controller, MAX10 Build version D.2.0.7
//****** PHY ******//
Object Id                     : 0xEC00000
PCIe s:b:d.f                  : 0000:08:00.0
Device Id                     : 0x0b30
Numa Node                     : 0
Ports Num                     : 01
Bitstream Id                  : 0x23000110010310
Bitstream Version             : 0.2.3
Pr Interface Id               : f3c99413-5081-4aad-bced-07eb84a6d0bb
//****** PHY GROUP 0 ******//
Direction                     : Line side
Speed                         : 25 Gbps
Number of PHYs                : 4
//****** PHY GROUP 1 ******//
Direction                     : Host side
Speed                         : 40 Gbps
Number of PHYs                : 4
//****** Intel C827 Retimer ******//
Port0 25G                     : Down
Port1 25G                     : Down
Port2 25G                     : Down
Port3 25G                     : Down
Retimer A Version             : 101c.1064
Retimer B Version             : 0000.0000

I suspect this is a cable issue because all four cards in our testbed show the same symptom. Is there a particular spec for a QSFP cable that is compatible with this card?

By the way, if I understand correctly, the default mode of BD-NVV-N3000-3 is 4x25GbE, where only one of the QSFP cage (i.e., cage A) is active. In this case, when we connect the Ethernet cable from QSFP cage A to cage B, do you still expect that the ports are up if there's no cable or card issue?

Regarding the loopback module, would the one in the following link work for this card?:

https://cablesondemand.com/qsfp-100g-loopback-adapter-module-for-qsfp28-port-testing-3db-attenuation-2-5w-power-consumption-100-gigabit-ethernet-ready-sf-100glb2-5w-3db?utm_source=google&utm_medium=cpc&adpos&scid=scplpSF-100GLB2.5W-3DB&sc_intid=SF-100GLB2.5W-3DB&gclid=Cj0KCQjwi7yCBhDJARIsAMWFScMuBbJujTg3weD0ZE1Z7H88h9qr_b3eVW1sUXOmBlNcgBSjR58T0TQaAs2_EALw_wcB

Thanks,

Daehyeok

    • Daehyeok_Kim's avatar
      Daehyeok_Kim
      Icon for New Contributor rankNew Contributor
      5 years ago

      Thanks for your reply!

      I switched the card configuration to 2x2x25G, tested it with cable from Cage A to Cage B, and checked that the port status is up:

      [root@h12 daehyeok]# fpgainfo phy
Board Management Controller, MAX10 NIOS FW version D.2.1.24
Board Management Controller, MAX10 Build version D.2.0.7
//****** PHY ******//
Object Id                     : 0xF100000
PCIe s:b:d.f                  : 0000:08:00.0
Device Id                     : 0x0b30
Numa Node                     : 0
Ports Num                     : 01
Bitstream Id                  : 0x23000410010310
Bitstream Version             : 0.2.3
Pr Interface Id               : a5d72a3c-c8b0-4939-912c-f715e5dc10ca
//****** PHY GROUP 0 ******//
Direction                     : Line side
Speed                         : 25 Gbps
Number of PHYs                : 4
//****** PHY GROUP 1 ******//
Direction                     : Host side
Speed                         : 40 Gbps
Number of PHYs                : 4
//****** Intel C827 Retimer ******//
Port0 25G                     : Up
Port1 25G                     : Up
Port2 25G                     : Up
Port3 25G                     : Up
Retimer A Version             : 101c.1064
Retimer B Version             : 101c.1064

      With this, I've confirmed that there's no issue with the cable and card.

      Since even in 2x2x25G mode, it still didn't work when I connected the card with the switch, I suspect there might be some issue with the configuration on the switch side. When I configured the switch port connected to the card, I set the rate to 25G and the FEC mode to RS for each lane (I checked the card is configured to use RS).

      Could you advise what else should I check on the switch side?

      Thanks,

      Daehyeok

      • JonWay_altera's avatar
        JonWay_altera
        Icon for Frequent Contributor rankFrequent Contributor
        5 years ago

        Hi @Daehyeok_Kim

        Thanks for your update on your debug status. Im glad to know that we have ruled out the cable or card issue. For the switch related queries, I am not the subject matter expert. Could you file a separate forum thread on the Ethernet products Forum (https://community.intel.com/t5/Ethernet-Products/bd-p/ethernet-products). They will be able to guide you from there. Appreciate if you can put in the link to the new thread for the benefit of others who face similar issue.