Our host PC can't discover Stratix 10 MX devkit as a PCIe device, even after a successful configuration and a proper installation of Linux driver.

Hi,

If you haven't done so, I would suggest you refer to the AN881 of the example design for the S10 MX board. Please refer to the following link:

https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/an/an881.pdf

Regards -SK

Thank you for your answer.

I've been suffered quite a long time with this problem. Having help from Intel is grateful and I am glad to talk to you.

But you should read my post at least in some important points, like environmental info. That five lines of comment have information that someone might be interested in if he really wants to help.

I clearly stated that I have tested the AN881 example design. I followed instructions there and it is not working even after I followed instruction exactly as the manual described. We've also consulted "Avalon-MM Intel Stratix 10 Hard IP+ for PCI Express Solutions User Guide"(UG-20170) because it contains the necessary information to install a Linux driver for PCIe communication.

I know Intel's new "24 hour quick response policy" that kicked in recently, but please, read before you post anything.

Best, HSuh.

Besides, since power may be a concern on the host server. Could you please use the external power adapter to power up the FPGA MX development kit and see if there is any difference.

Regards -SK

Hi, SengKokl.

I am sorry about my former post. I didn't know that you read my former post. I just checked my post frequently to make things clear. I thought that way a reader can grasp what's going on on my side easily.

Sorry for the confusion and for my attitude.

I will try to test your suggestions. Allow me to take some of my time because our server is located in the datacenter. To do the test you mentioned above, we need to once get the host server back to our office.

Meanwhile, could you tell me if there is a way to check that the settings(BIOS, physical slot placement on the motherboard, PCIe generation mode settings, etc.) on the host server is correct?

Since we don't know what's causing the problem right now, there is a chance that both FPGA configuration bitstream(.sof file) and the host server setting are wrong.

Currently, our FPGA is sitting on the x16 PCIe lane (gen 3.0) and we verified that the slot has full speed access to the host.

ASPM(Active State Power Management) level is disabled so that we can have full compatibility on any PCIe device.

Also, 4G decoding or Memory Mappinng of IO above 4GB is enable for the same reason. ( It allow 64bit PCI devices to be decoded above 4GB address space.)

Thank you,

Hsuh.

This is answers to your question.

1. No. Are you referring scripts that comes in the form of zip archive? We updated IPs so that it can be matched with Quartus19.2
2. Our device variant is 1SM21BHU2F53E2VGS1 not just E1VG. This is a ES variant.
3. Yes. It is compiled currectly and loaded before we run any test. (We load the driver again if the host is restarted)
4. No. I can try, but our IT policy allows us to use specific Linux distribution. We are using NFS system.
5. I will get back to you once I got the signal tap results.

As long as the slot of the Host PC can support Gen3X16, then it should be able to at least detect the FPGA’s PCIe interface (provided it can link up correctly). I don’t aware if any BIOS settings may impact it. The same reference design was tested here and which is working fine in one of our PC. You may try the same with different PC/server/slots to isolate if there is any dependency, and then you can also compare the BIOS setting for any difference.

Yes, please refer to the script.zip for the post-processing script that comes with the reference design. You need to run it to meet the requirement.

Regards -SK

Hi, SK.

I got a result from AN881 example design. I changed power supply to the one that comes with the Stratix 10, but it's still not working. Though, I think we need to use them, because when a host PC detects FPGA card, it enumerates the device list during POST and power will be reset once or twice. This power reset will erase FPGA configuration if the board is powering from the PSU of host PC.

Also, I tried to capture ltssmstate[5:0], currentspeed[1:0], lane_act[4:0] and link_up signals, but it was not successful. Is there any specific way to make it visible in the design? I know the original PCIe Hard IP+ has these signals, and I also turned on "Enable hard IP status bus when using AVMM interface," but the signals are still not visible.

So, I tried to turn on PCIe inspector by turning on PCIe instpector related options in IP parameter editor through Configuration Debug, and Extension options tab. However, it is getting errors with AN881 example design. (PFA. I attached screenshot for this.)

To find out what's causing the problem, I created an example design of original PCIe HARD IP+, and I got a PCIe test result from PCIe inspector. (Also attached this to the post). But I still can't see ltssm, currentspeed, etc. signals on the top level. I am experiencing these two problems.

1. ltssmstate, currentspeed, lane_act signals are not active and not exist.
2. signaltap node finder is not working. Even if there is signals, I can't search any node. I tested it with signals that confirmed as exist. It always says "no nodes found." I set "pcie_example_design.qsys" as a top in Quartus.

Is there any specific way to make those debug signals visible? Instead, I got you a result from PCIe inspector. THIS RESULT IS FOR PCIE HARD IP+'S EXAMPLE DESIGN. NOT FROM AN881.

Thank you.

Regards, Han.

Hi,

The PCIe Inspector result shows that it is Gen3x4, and the LTSSM status is L0. For the LTSSM signal, it is able to find from the signal tap by this setting (see attachment).

Regards -SK

I got the result. Seems like we can't set up the link in gen3 x16. These are the screenshot of Signal Tap analyzer for ltssm, link_up, currentspeed, etc.

THIS IS A SIGNALTAP INFO FOR AN881 EXAMPLE.

THE DEVICE IS ON PCIE SLOT4.

The signal keeps changing sequentially, which doesn't seem correct, or doing something meaningful inside of it. Signal sequence looks like this.

ltssmstate : 00h 08h 00h 08h 08h 00h 08h 00h 08h 00h 02h

currentspeed : 1h 1h 1h 1h 1h 1h 1h 1h 1h 1h 1h

lane_act : 10h 10h 10h 04h 10h 10h 10h 10h 10h 10h 10h

link_up : 0(always)

In this test, the FPGA board is in gen1 x16 state, but link is not setup.

We also tested FPGA board in other slots but it was not successful. We've tested from slot1 (direct connect to the CPU) to slot4. We couldn't test slot5 because there is no space that can fit the height of FPGA board in to the host PC's casing.

(slot5 is not a PCI slot. we have slot6 which is a PCI slot.) According to the manufacturer of the motherboard, we can use either slot2 or slot4 for full lane PCIe communiation(16 lane)

These are the test result from other slots. There was no transition in values like we had in slot4.

slot1 : (can't pass BIOS POST. Not bootable host PC) signaltap says ltssm(11h), currentspeed(03h), lane_act(04h), link_up(1)

slot2 : (booting of PC was successful) ltssm(11h), currentspeed(03h), lane_act(04h), link_up(1)

slot3 : (bootable) ltssm(h03), currentspeed(01h), lane_act(01h), link_up(0)

I have been thinking about it and I think FPGA board needs to be reset (without losing configuration) and need to kick in everything when host PC is powered up. I think PCIe IP just pass through initial handshaking status after bitstream(.sof) configuration.

Since FPGA board is now powered on with separate power supply(Intel's power), some sort of reset might do this. Do you have any suggestion? Or do you know what's happening on FPGA?

Thank you

Regards,

HSuh.

P.S. : This is our BIOS screenshot. Our host PC doesn't even see FPGA board as a PCIe device. As I said, it's currently sitting in slot4.

(This screenshot is taken after a configuration and after an enumeration.)