Forum Discussion

swm's avatar
swm
Icon for New Contributor rankNew Contributor
3 years ago

How to reliabily program multiple FPGAs using quartus_pgm?

I have a server with 8 x Stratix 10 SX boards (Terasic DE10Pro), which I would like to be able to program over usb-to-jtag. When I have one or two baords installed, I can reliably program a board, e...
FakhrulA_altera's avatar
FakhrulA_altera
Icon for Regular Contributor rankRegular Contributor
3 years ago

Hi swm,


Understood, I am checking this internally for more clarification.


I'd appreciate it if you could provide me with more information such as:

1. Could you please provide the screenshots of the project; Programmer; JTAG configuration etc?

2. Screenshots of the error message

3. Version of Quartus


Just to let you know that it might take some time, will get to you once there are any findings.


Regards,

Fakhrul



swm's avatar
swm
Icon for New Contributor rankNew Contributor
3 years ago

Hello Fakhrul,

I'm using Quartus 21.3Pro at present. I have 8 x Terasic DE10Pro cards that contain a Stratix 10 SX part and an on-board USB Blaster II that I'm using to program them in FPGA mode only (i.e. I'm not using the HPS at present). The details of the FPGA image don't appear to matter. In fact the specific evaluation card doesn't appear to matter either: we've seen similar problems programming several Stratix V boards, e.g. the Terasic DE5.

The DE10Pro boards are in a GPU-style server (a dual-socket AMD Epyc running Ubuntu 20.04.4 LTS) plugged into PCIe slots. The USB tree is as follows: 2 x native USB3 boards on the motherboard each connected to a four-port USB hub. Four boards are plugged into one hub and four into the other.

I'm programming the boards from the command-line, e.g.:

quartus_pgm -m jtag -c "DE10-Pro [5-2.3.1]" -o "p;../output_files/DE10_Pro.sof@1"

I've since done a bit of Python scripting (attached, with .py replaced with .txt so that it woudl upload) to monitor the output of quartus_pgm and retry programming on any errors, which at least gives me a good way forward. When using this script to reprogram all 8 boards 100 times I'm seeing around 10 to 20 retries, which isn't too bad. An example error report from quartus_pgm is also attached (quartus_pgm_error.txt). While this is probably now good enough that we can use the systems for automated regression test, etc., it is still curious that there are some failures.

I also notice that quartus_pgm checked for licenses but doesn't require a license to run. Making LM_LICENSE_FILE environment variable to an empty string causes quartus_pgm to not check for a license, which speeds things up.

- Simon