Daehyeok_Kim
New ContributorJoined 7 years ago
User Profile
New Contributor
User Widgets
Contributions
Re: Link status on BD-NVV-N3000-3 card
Thanks for your reply. I figured out what causes the issue. Since the factory/user image on FPGA does not support auto-negotiation, I had to manually disable auto-negotiation on the switch side. Otherwise, it seems to use auto-negotiation by default, and something went wrong. Thanks for your help! Daehyeok3KViews0likes0CommentsRe: Link status on BD-NVV-N3000-3 card
Thanks for your reply! I switched the card configuration to 2x2x25G, tested it with cable from Cage A to Cage B, and checked that the port status is up: [root@h12 daehyeok]# fpgainfo phy Board Management Controller, MAX10 NIOS FW version D.2.1.24 Board Management Controller, MAX10 Build version D.2.0.7 //****** PHY ******// Object Id : 0xF100000 PCIe s:b:d.f : 0000:08:00.0 Device Id : 0x0b30 Numa Node : 0 Ports Num : 01 Bitstream Id : 0x23000410010310 Bitstream Version : 0.2.3 Pr Interface Id : a5d72a3c-c8b0-4939-912c-f715e5dc10ca //****** PHY GROUP 0 ******// Direction : Line side Speed : 25 Gbps Number of PHYs : 4 //****** PHY GROUP 1 ******// Direction : Host side Speed : 40 Gbps Number of PHYs : 4 //****** Intel C827 Retimer ******// Port0 25G : Up Port1 25G : Up Port2 25G : Up Port3 25G : Up Retimer A Version : 101c.1064 Retimer B Version : 101c.1064 With this, I've confirmed that there's no issue with the cable and card. Since even in 2x2x25G mode, it still didn't work when I connected the card with the switch, I suspect there might be some issue with the configuration on the switch side. When I configured the switch port connected to the card, I set the rate to 25G and the FEC mode to RS for each lane (I checked the card is configured to use RS). Could you advise what else should I check on the switch side? Thanks, Daehyeok3KViews0likes2CommentsRe: Link status on BD-NVV-N3000-3 card
Thanks! We just checked the port status by connecting two cages using an Ethernet cable, but "fpgainfo phy" command still shows they are down: Board Management Controller, MAX10 NIOS FW version D.2.1.24 Board Management Controller, MAX10 Build version D.2.0.7 //****** PHY ******// Object Id : 0xEC00000 PCIe s:b:d.f : 0000:08:00.0 Device Id : 0x0b30 Numa Node : 0 Ports Num : 01 Bitstream Id : 0x23000110010310 Bitstream Version : 0.2.3 Pr Interface Id : f3c99413-5081-4aad-bced-07eb84a6d0bb //****** PHY GROUP 0 ******// Direction : Line side Speed : 25 Gbps Number of PHYs : 4 //****** PHY GROUP 1 ******// Direction : Host side Speed : 40 Gbps Number of PHYs : 4 //****** Intel C827 Retimer ******// Port0 25G : Down Port1 25G : Down Port2 25G : Down Port3 25G : Down Retimer A Version : 101c.1064 Retimer B Version : 0000.0000 I suspect this is a cable issue because all four cards in our testbed show the same symptom. Is there a particular spec for a QSFP cable that is compatible with this card? By the way, if I understand correctly, the default mode of BD-NVV-N3000-3 is 4x25GbE, where only one of the QSFP cage (i.e., cage A) is active. In this case, when we connect the Ethernet cable from QSFP cage A to cage B, do you still expect that the ports are up if there's no cable or card issue? Regarding the loopback module, would the one in the following link work for this card?: https://cablesondemand.com/qsfp-100g-loopback-adapter-module-for-qsfp28-port-testing-3db-attenuation-2-5w-power-consumption-100-gigabit-ethernet-ready-sf-100glb2-5w-3db?utm_source=google&utm_medium=cpc&adpos&scid=scplpSF-100GLB2.5W-3DB&sc_intid=SF-100GLB2.5W-3DB&gclid=Cj0KCQjwi7yCBhDJARIsAMWFScMuBbJujTg3weD0ZE1Z7H88h9qr_b3eVW1sUXOmBlNcgBSjR58T0TQaAs2_EALw_wcB Thanks, Daehyeok3KViews0likes4CommentsRe: Link status on BD-NVV-N3000-3 card
Thanks for your reply! I have some follow-up questions. 1) I am using BD-NVV-N3000-3 and haven't programmed the board with other images. I followed the instruction up to section 4 (https://www.intel.com/content/www/us/en/programmable/documentation/zsf1588015530773.html#phn1588111278439). Could you let me know how to verify which version is loaded? Also, what does OPN stands for? 2) By the "loopback module", are you referring to the module shown in Fig. 4 (https://www.intel.com/content/www/us/en/programmable/documentation/zsf1588015530773.html#xzd1588094394268)? Is it included in the original package? Thanks, Daehyeok3.1KViews0likes6CommentsLink status on BD-NVV-N3000-3 card
Hello, I've installed a BD-NVV-N3000-3 card on our server and connected it to an Intel Tofino-based 32-port 100Gbps switch using a QSFP+ cable. And I've followed the instruction in the user guide (https://www.intel.com/content/www/us/en/programmable/documentation/zsf1588015530773.html) to install the required runtime stack. I am trying to use it as a basic NIC with a 4 x 25G configuration. When I checked the status of Retimer using "sudo fpgainfo phy", it showed that the all four ports are down: -bash-4.2$ sudo fpgainfo phy Board Management Controller, MAX10 NIOS FW version D.2.1.24 Board Management Controller, MAX10 Build version D.2.0.7 //****** PHY ******// Object Id : 0xEC00000 PCIe s:b:d.f : 0000:08:00.0 Device Id : 0x0b30 Numa Node : 0 Ports Num : 01 Bitstream Id : 0x23000110010310 Bitstream Version : 0.2.3 Pr Interface Id : f3c99413-5081-4aad-bced-07eb84a6d0bb //****** PHY GROUP 0 ******// Direction : Line side Speed : 25 Gbps Number of PHYs : 4 //****** PHY GROUP 1 ******// Direction : Host side Speed : 40 Gbps Number of PHYs : 4 //****** Intel C827 Retimer ******// Port0 25G : Down Port1 25G : Down Port2 25G : Down Port3 25G : Down Retimer A Version : 101c.1064 Retimer B Version : 0000.0000 On the switch side, I could see the port status for all four lanes is down: -----+----+---+----+-------+----+---+---+---+--------+----------------+----------------+- PORT |MAC |D_P|P/PT|SPEED |FEC |RDY|ADM|OPR|LPBK |FRAMES RX |FRAMES TX |E -----+----+---+----+-------+----+---+---+---+--------+----------------+----------------+- 25/0 |31/0|188|3/60|25G | RS |YES|ENB|DWN| NONE | 0| 0| 25/1 |31/1|189|3/61|25G | RS |YES|ENB|DWN| NONE | 0| 0| 25/2 |31/2|190|3/62|25G | RS |YES|ENB|DWN| NONE | 0| 0| 25/3 |31/3|191|3/63|25G | RS |YES|ENB|DWN| NONE | 0| 0| Could you let me know how to activate the ports? Please let me know if you need any further information. Thanks, DaehyeokSolved3.2KViews0likes8CommentsSystem crashes when installing BD-NVV-N3000-3 on Dell R720
Hello, We're trying to install the BD-NVV-N3000-3 card on our Dell R720 machine. We installed the card to one of x16 PCIe slots and connected it with a 6-pin power connector. The system can boot with the card installed and it shows up in lspci output, but after a very short time - usually a matter of minutes - the system crashes with this log: h11 login: [ 174.287195] {1}[Hardware Error]: Hardware error from APEI Generic Hardware Error Source: 32992 [ 174.296815] {1}[Hardware Error]: event severity: fatal [ 174.302550] {1}[Hardware Error]: Error 0, type: fatal [ 174.308285] {1}[Hardware Error]: section_type: PCIe error [ 174.314504] {1}[Hardware Error]: port_type: 4, root port [ 174.320625] {1}[Hardware Error]: version: 1.0 [ 174.325682] {1}[Hardware Error]: command: 0x0547, status: 0x4010 [ 174.332579] {1}[Hardware Error]: device_id: 0000:40:02.0 [ 174.338698] {1}[Hardware Error]: slot: 0 [ 174.343266] {1}[Hardware Error]: secondary_bus: 0x42 [ 174.348999] {1}[Hardware Error]: vendor_id: 0x8086, device_id: 0x0e04 [ 174.356381] {1}[Hardware Error]: class_code: 000406 [ 174.362018] {1}[Hardware Error]: bridge: secondary_status: 0x2000, control: 0x0003 [ 174.370662] Kernel panic - not syncing: Fatal hardware error! [ 174.377153] Kernel Offset: 0x3d600000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) [ 174.391984] Rebooting in 30 seconds.. We noticed that this can happen even when the kernel is not running, like if you're sitting at a grub prompt or even during the UEFI pre-boot environment (see attached log). We tried the card in each of the x16 slots and also tried removing the other adapters but to no avail. Does anyone know why this happens? Please let me know if you need any further information. Thanks, DaehyeokSolved2KViews0likes2Comments